-- Logs begin at Tue 2022-07-26 03:43:49 UTC, end at Tue 2022-07-26 05:19:54 UTC. -- Jul 26 04:45:12.161097 dv-cinder-850574-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 26 04:45:12.170747 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 26 04:45:12.172922 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 26 04:45:12 2022] *** Jul 26 04:45:12.172922 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 26 04:45:12.172922 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 26 04:45:12.172922 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: nodename: dv-cinder-850574-1-iscsi Jul 26 04:45:12.172922 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: machine: x86_64 Jul 26 04:45:12.172922 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: clock source: unix Jul 26 04:45:12.172922 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: pcre jit disabled Jul 26 04:45:12.172922 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: detected number of CPU cores: 4 Jul 26 04:45:12.172922 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: current working directory: / Jul 26 04:45:12.172922 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: detected binary path: /usr/bin/uwsgi-core Jul 26 04:45:12.172922 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: your processes number limit is 47680 Jul 26 04:45:12.172922 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: your memory page size is 4096 bytes Jul 26 04:45:12.173736 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: detected max file descriptor number: 2048 Jul 26 04:45:12.173736 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: lock engine: pthread robust mutexes Jul 26 04:45:12.173736 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: thunder lock: enabled Jul 26 04:45:12.173736 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 26 04:45:12.173736 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 26 04:45:12.213464 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: Python main interpreter initialized at 0x55e6919dd9c0 Jul 26 04:45:12.213464 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: python threads support enabled Jul 26 04:45:12.213464 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: your server socket listen backlog is limited to 100 connections Jul 26 04:45:12.213464 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: your mercy for graceful operations on workers is 90 seconds Jul 26 04:45:12.213894 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: mapped 403077 bytes (393 KB) for 2 cores Jul 26 04:45:12.213894 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: *** Operational MODE: preforking *** Jul 26 04:45:12.214248 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: *** uWSGI is running in multiple interpreter mode *** Jul 26 04:45:12.214248 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: spawned uWSGI master process (pid: 80193) Jul 26 04:45:12.214442 dv-cinder-850574-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 26 04:45:12.215589 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: spawned uWSGI worker 1 (pid: 80205, cores: 1) Jul 26 04:45:12.215589 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: spawned uWSGI worker 2 (pid: 80206, cores: 1) Jul 26 04:45:12.215745 dv-cinder-850574-1-iscsi devstack@keystone.service[80193]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 26 04:45:17.745141 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 04:45:17.745141 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: from cryptography.utils import int_from_bytes Jul 26 04:45:17.746154 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 04:45:17.746154 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: from cryptography.utils import int_from_bytes Jul 26 04:45:17.754389 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 04:45:17.754389 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: from cryptography.utils import int_from_bytes Jul 26 04:45:17.754948 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 04:45:17.754948 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: from cryptography.utils import int_from_bytes Jul 26 04:45:17.795496 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 26 04:45:17.795765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 26 04:45:17.795893 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] command line args: [] {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 26 04:45:17.796159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 26 04:45:17.796338 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ================================================================================ {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 26 04:45:17.796598 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] admin_token = **** {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.796841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] config_dir = [] {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.797025 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.797203 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] config_source = [] {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.797382 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] debug = True {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.797578 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.797975 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.798169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.798377 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.798624 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.798861 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] list_limit = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.799037 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] log_config_append = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.799200 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.799395 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] log_dir = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.799607 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] log_file = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.799783 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.800012 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.800256 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.800507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.800685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.800862 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.801104 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.801345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.801576 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.801808 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.802039 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.802268 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.802519 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.802760 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.803033 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.803504 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 26 04:45:17.803748 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.803873 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 26 04:45:17.804054 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] publish_errors = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.804184 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] command line args: [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 26 04:45:17.804368 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.804480 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 26 04:45:17.804654 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.804777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ================================================================================ {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 26 04:45:17.804899 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.805011 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] admin_token = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.805123 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.805123 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.805408 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] config_dir = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.805408 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.805687 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] standard_threads = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.805803 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] config_source = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.805916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.806035 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] debug = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.806174 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.806303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.806467 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.806582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.806693 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] use_journal = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.806826 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.806956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] use_json = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.806956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] use_stderr = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.807157 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.807587 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] use_syslog = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.807703 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.807856 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.807984 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.808071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.808141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_config_append = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.808141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.808317 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.808399 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_dir = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.808511 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.808583 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_file = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.808697 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.808769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.808883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.808963 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.809072 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.809144 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.809250 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.809340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.809340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.809517 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.809590 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.809699 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.809778 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.809940 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.810020 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.810130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.810209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.810333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.810450 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.810574 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.810687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.810792 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.810900 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.811010 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.811130 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.811258 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.811372 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.811492 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.811634 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.811749 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.811749 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.811973 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.812083 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.812206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] publish_errors = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.812331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.812454 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.812568 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.812682 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.812803 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.812917 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.813045 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.813164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.813273 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.813389 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.813498 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.813611 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] standard_threads = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.813738 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.813852 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.814062 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.814179 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.814179 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.814415 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.814538 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] use_journal = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.814660 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.814784 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] use_json = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.814912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.815024 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] use_stderr = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.815024 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] use_syslog = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.815348 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.815463 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 04:45:17.815594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.815694 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.815799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.815897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.816010 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.816119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.816226 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.816330 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.816426 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.816426 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.816629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.816739 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.816847 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.816847 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.817058 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.817164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.817164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.817375 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.817482 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.817482 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.817681 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.817794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.817794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.818017 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.818121 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.818121 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.818321 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.818430 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.818430 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.818638 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.818753 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.818753 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.818959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.819070 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.819191 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.819304 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.819304 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.819505 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.819631 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.819762 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.819885 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.819988 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.820144 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.820261 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.820366 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.820593 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.820702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.820819 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.821069 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.821175 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.821316 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.821546 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.821655 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.821767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.822015 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.822136 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.822263 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.822488 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.822809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.822809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.823016 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.823124 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.823238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.823342 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.823436 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.823672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.823799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.823902 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.823994 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.824099 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.824198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.824299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.824396 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.824535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.824691 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.824805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.824899 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.824994 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.825225 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.825330 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.825459 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.825579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.825688 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.825791 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.825895 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.826002 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.826153 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.826286 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.826405 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.826576 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.826702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.826818 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.826923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.827061 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.827186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.827294 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.827397 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.827502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.827640 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.827746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.827999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.828121 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.828243 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.828402 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.828527 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.828683 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.828808 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.828938 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.829157 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.829270 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.829390 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.829573 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.829687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.829818 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.829948 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.830064 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.830192 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.830307 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.830424 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.830552 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.830668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.830782 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.830888 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.831001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.831224 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.831374 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.831647 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.831769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.831910 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.832016 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.832151 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.832334 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.832448 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.832679 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.832790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.832929 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.833052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.833158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.833264 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.833399 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.833635 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.833744 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.833939 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.834045 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.834163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.834322 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.834433 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.834541 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.834664 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.834782 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.834904 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.835010 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.835119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.835268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.835385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.835491 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.835758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.835870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.836001 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.836219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.836338 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.836445 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.836656 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.836768 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.836877 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.837107 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.837233 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.837339 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.837580 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.837688 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.837829 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.838040 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.838149 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.838316 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.838479 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] auth.external = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.838586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.838808 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.838914 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.839047 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.external = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.839282 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.839539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.839768 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.839893 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.840001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.840111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.password = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.840335 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.token = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.840442 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] auth.password = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.840818 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.840927 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] auth.token = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.841043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.841043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.841248 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.841355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.841479 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.841617 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.841727 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.841948 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.842081 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.842188 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.842422 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.842530 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.842676 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.842926 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.843034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.843166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.843399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.843509 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.843653 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.843861 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.843971 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.844157 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.844281 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] credential.caching = True {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.844388 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.844633 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.844742 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.844927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.845133 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.845303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.845577 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.845692 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.845805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.845947 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.846056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.846291 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.846399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.846512 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.846618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.846768 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.846882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.847017 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.847603 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.847717 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.847857 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.848037 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.848146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.848353 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.848460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.848597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.848710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.848854 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.848963 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.849088 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.849332 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.849441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.849553 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.849692 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.849798 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.849923 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.850033 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.850145 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] federation.caching = True {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.850256 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.850472 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.850853 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.850853 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.851074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.851196 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.851310 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.851422 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.851533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.851828 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.851936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.852057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.852057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.852266 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.852379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.852379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.852605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.852721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.852837 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.852954 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.853074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.853200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.853314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.853430 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.853546 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.853708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.853830 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.853985 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.854162 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] identity.caching = True {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.854272 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.854446 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.854640 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.854762 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.854902 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.855120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.855230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.855371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.855613 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.855742 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.855878 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.856063 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.856172 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.856347 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.856539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.856660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.856818 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.856989 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.857099 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.857300 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.857450 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.857570 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.857765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.857940 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.858050 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.858222 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.858424 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.858545 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.858699 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.859111 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.859226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.859226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.859428 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.859574 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.859696 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.859810 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.859927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.860099 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.860263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.860390 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.860582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.860777 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.860889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.861073 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.861261 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.861385 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.861529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.861763 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.861874 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.862024 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.862247 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.862369 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.862478 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.862727 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.862836 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.862975 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.863211 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.863382 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.863500 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.863689 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.863792 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.863907 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.864137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.864264 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.864372 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.864558 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.864669 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.864827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.864937 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.865060 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.865167 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.865306 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.865437 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.865557 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.865664 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.865787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.865894 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.866005 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.866117 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.866229 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.866340 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.866468 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.866576 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.866692 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.866802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.866914 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.867029 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.867153 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.867261 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.867374 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.867486 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.867621 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.867869 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.867999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.868117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.868260 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.868371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.868552 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.868673 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.868788 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.868997 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.869107 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.869254 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.869406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.869591 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.869701 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.869879 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.869987 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.870103 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.870353 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.870473 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.870590 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.870845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.870953 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.871075 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.871298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.871420 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.871602 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.871765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.871875 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.872059 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.872234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.872362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.872526 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.872705 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.872814 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] role.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.872999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] role.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.873154 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.873276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.873469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.873605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.873960 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.874069 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.874230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.874471 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.user = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.874580 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.874720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.874927 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.875037 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.875203 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.875413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.875524 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.875703 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.875896 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.876007 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.876191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.876425 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.876538 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.876731 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.876880 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.877191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.877328 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.877450 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.877673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.877784 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.877925 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.878143 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.878255 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.878409 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.878552 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.878663 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.878891 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.879001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.879146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.879352 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.879463 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.879641 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.879779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.879889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.880096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.880205 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.880319 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.880552 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.880660 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.880803 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.881009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.881118 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.881266 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.881498 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.881606 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.881721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.881849 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.881959 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.882196 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.882304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.882428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.882538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.882663 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.882772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.882915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.883074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.883185 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.883386 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.883505 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.883644 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.883888 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.884000 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.884115 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.884354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.884478 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.884595 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.884819 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.884927 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.885069 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.885295 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.885417 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.885788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.885903 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.886089 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.886237 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.886351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.connection = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.886590 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.886715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.886844 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.887064 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.887178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.887301 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.887533 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.887677 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.887810 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.888056 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.888176 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.888340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.888487 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.888917 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.889128 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.889323 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.889442 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.889734 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.890022 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.890167 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] resource.caching = True {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.890291 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.890503 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.890619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.890734 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.890848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.890966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.891090 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.891200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.891318 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.891428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.891628 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.891760 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.891869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 26 04:45:17.891989 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.892126 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.892400 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.892701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.893107 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] role.caching = True {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.893555 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] role.driver = sql {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.893955 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.894410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.894890 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.895367 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.895830 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.896250 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.896690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.897084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.897383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.897629 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.897929 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.898354 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.898738 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.899154 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.899941 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.900395 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.900845 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.901319 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.901799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.902268 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.902715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.903190 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.903696 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.904158 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.904598 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.905036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.905466 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.905925 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.906407 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.906857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.907294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.907835 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.908277 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.908692 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] token.caching = True {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.909107 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.909589 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.910038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.910485 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.910916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.911326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.911777 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.912096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.912354 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.912605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.912849 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.913095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.913340 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.913591 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.913823 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.914068 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.914370 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.914629 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] database.connection = **** {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.914876 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.915125 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.915372 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.915645 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.915884 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.916123 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.916362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.916600 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.916847 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.917082 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.917326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.917561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.917806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.918039 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.918283 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.918512 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.918751 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.918983 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 04:45:17.919152 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80205) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 26 04:45:17.928749 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:17.928978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:17.929209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:17.937881 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:17.944440 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:17.944716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:17.944874 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:17.953497 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:17.968491 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:17.968783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:17.973784 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:17.974011 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.023837 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.024139 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.076452 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.076819 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.077149 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.077384 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.144824 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.145184 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.145406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.145656 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.145866 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.146078 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.146288 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.146496 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.146726 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.146959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.147167 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.147167 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.169513 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.169797 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.169939 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.170078 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.170205 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.170332 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.170459 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.170596 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.170729 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.170855 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.170978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.171160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.171414 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.171695 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.183321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.183659 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.196926 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.197208 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.197357 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.197491 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.197628 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.197768 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.197975 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.198160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.198349 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.198478 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.198629 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80205) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.198802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.198802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.198802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 04:45:18.215231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.215489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 04:45:18.215752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.215892 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.216060 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 04:45:18.216187 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 04:45:18.216313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.216435 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.216561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.216694 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 04:45:18.216834 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.216955 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.217074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.217196 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.217196 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.217457 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.227052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.227341 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.227732 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.227949 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.228194 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.228406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.228406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.228815 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.234883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80205) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.235539 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.239898 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80205) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.240182 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.240470 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.240672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.240906 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.241113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.241343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.241545 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.241747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.241949 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.242163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.242380 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.242579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.242782 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.242984 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.243194 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.243390 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.243640 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.243843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.244041 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.244041 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.244041 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.265447 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.266977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80205) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.267285 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.267582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.269066 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.269358 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.271878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80205) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.272155 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.272436 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.274047 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.274262 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.274546 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.278960 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80205) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.279240 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.281162 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.281440 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.283377 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80205) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.283692 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.283981 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.284252 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.284523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.284797 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.285068 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.285448 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.285635 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.285938 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.286213 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.286491 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.286760 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.287034 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.287320 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.287617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.301276 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.301570 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.303653 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.303947 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.305465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80205) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.307885 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.308592 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.308884 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.311014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.311319 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.313264 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80205) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.315776 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.316466 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80205) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.318946 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.319683 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.319985 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.320262 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.320543 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.320822 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.321107 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.322160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.322460 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.322746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.323037 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.323312 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.323616 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.337198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80205) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.337501 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.337768 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.338035 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.339963 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.340272 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.340540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.340801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.344947 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.346524 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.346904 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.348160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.349708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.349963 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80205) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.350074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.350243 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.350544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.350819 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.351098 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.352823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.353108 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.353400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.353686 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.353956 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.362599 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80205) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.362874 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.363152 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.363434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.363823 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.364029 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.364311 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.365534 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.365815 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.366092 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.366366 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.366645 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.366914 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.367195 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.379937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80205) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.382905 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80205) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.383359 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.386391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.387065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.389094 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.389467 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.390116 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80205) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.391004 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.391004 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.392266 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.393856 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.394141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.394423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.397606 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.399119 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80205) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.401154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.402112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.402409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.402683 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.402770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.402957 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.405725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.406018 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.406305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.406578 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.411022 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80205) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.411313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.411612 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.414718 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.415017 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.415293 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.418421 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80205) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.418696 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.418980 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.419369 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.419666 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.419946 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.420224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.420509 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.420774 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.421052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.421328 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.421591 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.421863 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.422115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80205) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.422275 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 04:45:18.422488 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.422707 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.422989 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.423364 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.423663 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.423940 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.424199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.424575 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.424865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.425144 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.425404 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.425665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.425920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 04:45:18.455649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e6919dd9c0 pid: 80205 (default app) Jul 26 04:45:18.459315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238495f7-814c-45b3-a3e4-33bb7e22526e None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:18.459473 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e6919dd9c0 pid: 80206 (default app) Jul 26 04:45:18.459653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238495f7-814c-45b3-a3e4-33bb7e22526e None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:18.459899 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238495f7-814c-45b3-a3e4-33bb7e22526e None None] PATH_INFO: `/v3` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:18.464141 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Tue Jul 26 04:45:18 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:26.323491 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e741295-cdb2-4c5f-bd44-2e3ee699173e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:26.324130 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e741295-cdb2-4c5f-bd44-2e3ee699173e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:26.324130 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e741295-cdb2-4c5f-bd44-2e3ee699173e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:26.328778 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1/2] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:26 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:26.335268 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd81d1d-7743-4e49-a891-9a84812f7875 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:26.335864 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd81d1d-7743-4e49-a891-9a84812f7875 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:26.335864 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd81d1d-7743-4e49-a891-9a84812f7875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:26.408109 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: INFO dbcounter [None req-2fd81d1d-7743-4e49-a891-9a84812f7875 None None] Registered counter for database keystone Jul 26 04:45:26.431409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG oslo_db.sqlalchemy.engines [None req-2fd81d1d-7743-4e49-a891-9a84812f7875 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80205) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 26 04:45:26.438564 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writer thread running {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 26 04:45:26.618889 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG passlib.handlers.bcrypt [None req-2fd81d1d-7743-4e49-a891-9a84812f7875 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80205) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 26 04:45:26.619295 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG passlib.handlers.bcrypt [None req-2fd81d1d-7743-4e49-a891-9a84812f7875 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80205) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 26 04:45:26.675350 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-2fd81d1d-7743-4e49-a891-9a84812f7875 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:26.677009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-2fd81d1d-7743-4e49-a891-9a84812f7875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:26.810039 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 2/3] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:26 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 26 04:45:26.946519 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e741295-cdb2-4c5f-bd44-2e3ee699173e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:27.015322 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: INFO dbcounter [None req-2e741295-cdb2-4c5f-bd44-2e3ee699173e None None] Registered counter for database keystone Jul 26 04:45:27.035062 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG oslo_db.sqlalchemy.engines [None req-2e741295-cdb2-4c5f-bd44-2e3ee699173e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80206) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 26 04:45:27.045583 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writer thread running {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 26 04:45:27.167385 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e741295-cdb2-4c5f-bd44-2e3ee699173e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:27.172368 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ee336-5c28-4361-811f-f21a1dc2bf0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:27.173127 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ee336-5c28-4361-811f-f21a1dc2bf0a admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:27.173412 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ee336-5c28-4361-811f-f21a1dc2bf0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:27.173693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ee336-5c28-4361-811f-f21a1dc2bf0a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:27.183302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ee336-5c28-4361-811f-f21a1dc2bf0a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:27.235409 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d96ee336-5c28-4361-811f-f21a1dc2bf0a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 26 04:45:27.235661 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ee336-5c28-4361-811f-f21a1dc2bf0a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:27.246529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 2/4] 10.222.0.17 () {58 vars in 1324 bytes} [Tue Jul 26 04:45:26 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 04:45:27.253058 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd81d1d-7743-4e49-a891-9a84812f7875 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:27.268428 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd81d1d-7743-4e49-a891-9a84812f7875 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:27.273216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a81bbd1-d53b-4f64-b3e0-cddb32caeb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:27.274367 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81bbd1-d53b-4f64-b3e0-cddb32caeb9d admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:27.274658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81bbd1-d53b-4f64-b3e0-cddb32caeb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:27.274940 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81bbd1-d53b-4f64-b3e0-cddb32caeb9d admin admin] PATH_INFO: `/v3/projects/d12823499382429e9825c57ea3292c56` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:27.276628 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a81bbd1-d53b-4f64-b3e0-cddb32caeb9d admin admin] RBAC: Authorizing `identity:get_project(project_id=d12823499382429e9825c57ea3292c56)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:27.328192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2a81bbd1-d53b-4f64-b3e0-cddb32caeb9d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 26 04:45:27.328523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a81bbd1-d53b-4f64-b3e0-cddb32caeb9d admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:27.333729 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 3/5] 10.222.0.17 () {56 vars in 1207 bytes} [Tue Jul 26 04:45:27 2022] GET /identity/v3/projects/d12823499382429e9825c57ea3292c56 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:29.381884 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401eac2-4e03-4d73-a17e-353b2f2e1eb1 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:29.382439 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401eac2-4e03-4d73-a17e-353b2f2e1eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:29.382439 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401eac2-4e03-4d73-a17e-353b2f2e1eb1 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:29.386629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 3/6] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:29.392868 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fed42e-f0da-4701-9735-ab52ac730415 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:29.393269 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fed42e-f0da-4701-9735-ab52ac730415 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:29.393430 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fed42e-f0da-4701-9735-ab52ac730415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:29.414419 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-34fed42e-f0da-4701-9735-ab52ac730415 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:29.416021 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-34fed42e-f0da-4701-9735-ab52ac730415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:29.424948 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 4/7] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:29 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 26 04:45:29.447181 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0401eac2-4e03-4d73-a17e-353b2f2e1eb1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:29.468727 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0401eac2-4e03-4d73-a17e-353b2f2e1eb1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:29.476449 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ab495-9462-4eef-a140-79335dd0a3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:29.477583 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ab495-9462-4eef-a140-79335dd0a3a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:29.478114 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ab495-9462-4eef-a140-79335dd0a3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:29.478636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ab495-9462-4eef-a140-79335dd0a3a3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:29.494387 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ab495-9462-4eef-a140-79335dd0a3a3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:29.497493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ab495-9462-4eef-a140-79335dd0a3a3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:29.506654 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 4/8] 10.222.0.17 () {58 vars in 1324 bytes} [Tue Jul 26 04:45:29 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 04:45:29.512847 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fed42e-f0da-4701-9735-ab52ac730415 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:29.522838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fed42e-f0da-4701-9735-ab52ac730415 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:29.528422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba27676-3d60-428e-9a4f-c8fb88e20f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:29.529077 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba27676-3d60-428e-9a4f-c8fb88e20f59 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:29.529355 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba27676-3d60-428e-9a4f-c8fb88e20f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:29.529624 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba27676-3d60-428e-9a4f-c8fb88e20f59 admin admin] PATH_INFO: `/v3/users/1baa40dae37f4b3298a95bdc939c2f42` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:29.531128 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba27676-3d60-428e-9a4f-c8fb88e20f59 admin admin] RBAC: Authorizing `identity:get_user(user_id=1baa40dae37f4b3298a95bdc939c2f42)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:29.533062 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba27676-3d60-428e-9a4f-c8fb88e20f59 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:29.534626 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 5/9] 10.222.0.17 () {56 vars in 1198 bytes} [Tue Jul 26 04:45:29 2022] GET /identity/v3/users/1baa40dae37f4b3298a95bdc939c2f42 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:45:31.655754 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ba515-57bc-46b0-8ba1-185cee4040c0 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:31.656263 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ba515-57bc-46b0-8ba1-185cee4040c0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:31.656263 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ba515-57bc-46b0-8ba1-185cee4040c0 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:31.657760 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 5/10] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:31.663209 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19984933-49fe-4ee0-8276-f68935c8e26b None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:31.663825 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19984933-49fe-4ee0-8276-f68935c8e26b None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:31.664098 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19984933-49fe-4ee0-8276-f68935c8e26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:31.673384 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701da71-7863-4351-9470-041de08c0aca None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:31.673660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701da71-7863-4351-9470-041de08c0aca None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:31.673923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701da71-7863-4351-9470-041de08c0aca None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:31.675308 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 6/11] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:31.681136 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1ccb7-bb7c-494e-91a7-29a90130d4d5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:31.681425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1ccb7-bb7c-494e-91a7-29a90130d4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:31.681702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1ccb7-bb7c-494e-91a7-29a90130d4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:31.684358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-19984933-49fe-4ee0-8276-f68935c8e26b None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:31.685563 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-19984933-49fe-4ee0-8276-f68935c8e26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:31.725112 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG passlib.handlers.bcrypt [None req-62e1ccb7-bb7c-494e-91a7-29a90130d4d5 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80206) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 26 04:45:31.725540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG passlib.handlers.bcrypt [None req-62e1ccb7-bb7c-494e-91a7-29a90130d4d5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80206) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 26 04:45:31.768296 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 6/12] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:45:31.770794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-62e1ccb7-bb7c-494e-91a7-29a90130d4d5 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:31.772460 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-62e1ccb7-bb7c-494e-91a7-29a90130d4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:31.798837 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19984933-49fe-4ee0-8276-f68935c8e26b None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:31.829775 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:31.883656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 7/13] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:45:31.906626 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1ccb7-bb7c-494e-91a7-29a90130d4d5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:31.913409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19984933-49fe-4ee0-8276-f68935c8e26b None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:31.948611 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3cd814-4596-4945-9f2e-14dec91516b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:31.949252 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3cd814-4596-4945-9f2e-14dec91516b4 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:31.949535 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3cd814-4596-4945-9f2e-14dec91516b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:31.949818 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3cd814-4596-4945-9f2e-14dec91516b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:32.010492 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1ccb7-bb7c-494e-91a7-29a90130d4d5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:32.012894 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:32.019760 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3cd814-4596-4945-9f2e-14dec91516b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:32.021304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3cd814-4596-4945-9f2e-14dec91516b4 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:32.045719 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:32.048692 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6803089-176d-4aaa-8943-9374f5c2bbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:32.049835 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6803089-176d-4aaa-8943-9374f5c2bbd9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:32.050123 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6803089-176d-4aaa-8943-9374f5c2bbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:32.050393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6803089-176d-4aaa-8943-9374f5c2bbd9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:32.062529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6803089-176d-4aaa-8943-9374f5c2bbd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:45:32.071028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c6803089-176d-4aaa-8943-9374f5c2bbd9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:45:32.071028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:45:32.071028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:45:32.071028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:45:32.071028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:45:32.071028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:45:32.071028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:45:32.071028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:45:32.071028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:45:32.071028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:45:32.071028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:45:32.071028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:45:32.071028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:45:32.071028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:45:32.071028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:45:32.071028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:45:32.072083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:45:32.072083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:45:32.072083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:45:32.072083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:45:32.072083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:45:32.072083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:45:32.072083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:45:32.072083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:45:32.072083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:45:32.072083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:45:32.072083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:45:32.072083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:45:32.072083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:45:32.072083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:45:32.072083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:45:32.072083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:45:32.072927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:45:32.072927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:45:32.072927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:45:32.072927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:45:32.072927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:45:32.072927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:45:32.072927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:45:32.072927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:45:32.072927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:45:32.072927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:45:32.072927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:45:32.072927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:45:32.072927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 8/14] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:45:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:45:32.078247 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6803089-176d-4aaa-8943-9374f5c2bbd9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:32.126492 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 7/15] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:45:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 04:45:32.132452 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3cd814-4596-4945-9f2e-14dec91516b4 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:32.179643 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6803089-176d-4aaa-8943-9374f5c2bbd9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:32.213730 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7911626-25f6-4dd7-b863-2e777e16984a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:32.214343 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7911626-25f6-4dd7-b863-2e777e16984a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:32.214625 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7911626-25f6-4dd7-b863-2e777e16984a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:32.214903 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7911626-25f6-4dd7-b863-2e777e16984a None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:32.215718 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7911626-25f6-4dd7-b863-2e777e16984a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:32.216166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7911626-25f6-4dd7-b863-2e777e16984a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:32.217441 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7911626-25f6-4dd7-b863-2e777e16984a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:32.230373 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 9/16] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:45:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:32.236223 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7911626-25f6-4dd7-b863-2e777e16984a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:32.239992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3cd814-4596-4945-9f2e-14dec91516b4 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:32.267828 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:32.354668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7911626-25f6-4dd7-b863-2e777e16984a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:32.376562 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:32.388365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33938ef2-8ffb-4022-8e00-6eb4c9ca552a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:32.389027 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33938ef2-8ffb-4022-8e00-6eb4c9ca552a None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:32.389316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33938ef2-8ffb-4022-8e00-6eb4c9ca552a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:32.389585 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33938ef2-8ffb-4022-8e00-6eb4c9ca552a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:32.393550 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a2a4b-7a09-4f1f-b88d-cb242a3427af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:32.394224 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a2a4b-7a09-4f1f-b88d-cb242a3427af None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:32.394513 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a2a4b-7a09-4f1f-b88d-cb242a3427af None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:32.394792 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a2a4b-7a09-4f1f-b88d-cb242a3427af None admin] PATH_INFO: `/v3/users/1baa40dae37f4b3298a95bdc939c2f42` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:32.396394 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a2a4b-7a09-4f1f-b88d-cb242a3427af None admin] RBAC: Authorizing `identity:get_user(user_id=1baa40dae37f4b3298a95bdc939c2f42)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:32.397886 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a2a4b-7a09-4f1f-b88d-cb242a3427af None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:32.399608 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 10/17] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:45:32 2022] GET /identity/v3/users/1baa40dae37f4b3298a95bdc939c2f42 => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:45:32.399978 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33938ef2-8ffb-4022-8e00-6eb4c9ca552a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:32.401458 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33938ef2-8ffb-4022-8e00-6eb4c9ca552a None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:32.405194 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a2a4b-7a09-4f1f-b88d-cb242a3427af None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:32.419643 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-33938ef2-8ffb-4022-8e00-6eb4c9ca552a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:45:32.419643 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:45:32.419643 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:45:32.419643 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:45:32.419643 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:45:32.419643 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:45:32.419643 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:45:32.419643 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:45:32.419643 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:45:32.419643 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:45:32.419643 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:45:32.419643 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:45:32.419643 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:45:32.419643 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:45:32.419643 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:45:32.420494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:45:32.420494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:45:32.420494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:45:32.420494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:45:32.420494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:45:32.420494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:45:32.420494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:45:32.420494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:45:32.420494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:45:32.420494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:45:32.420494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:45:32.420494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:45:32.420494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:45:32.420494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:45:32.420494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:45:32.420494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:45:32.420494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:45:32.421282 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:45:32.421282 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:45:32.421282 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:45:32.421282 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:45:32.421282 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 8/18] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:45:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:45:32.427070 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33938ef2-8ffb-4022-8e00-6eb4c9ca552a None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:32.504410 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:32.512910 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a2a4b-7a09-4f1f-b88d-cb242a3427af None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:32.530808 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33938ef2-8ffb-4022-8e00-6eb4c9ca552a None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:32.546488 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02717690-7a8f-4c6e-9e6a-2c1466f65bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:32.547146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02717690-7a8f-4c6e-9e6a-2c1466f65bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:32.547429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02717690-7a8f-4c6e-9e6a-2c1466f65bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:32.547895 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02717690-7a8f-4c6e-9e6a-2c1466f65bb9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:32.549067 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02717690-7a8f-4c6e-9e6a-2c1466f65bb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:32.550543 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02717690-7a8f-4c6e-9e6a-2c1466f65bb9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:32.552471 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 11/19] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:45:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:32.558050 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02717690-7a8f-4c6e-9e6a-2c1466f65bb9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:32.565104 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44368249-457e-4905-a19a-a42af43d3e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:32.565745 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44368249-457e-4905-a19a-a42af43d3e10 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:32.566030 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44368249-457e-4905-a19a-a42af43d3e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:32.566300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44368249-457e-4905-a19a-a42af43d3e10 None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:32.567014 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44368249-457e-4905-a19a-a42af43d3e10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:32.567322 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44368249-457e-4905-a19a-a42af43d3e10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:32.568755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44368249-457e-4905-a19a-a42af43d3e10 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:32.587073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 9/20] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:45:32.676560 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02717690-7a8f-4c6e-9e6a-2c1466f65bb9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:32.714124 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540540c-79dc-4b20-8cbc-7f085333d29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:32.714777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540540c-79dc-4b20-8cbc-7f085333d29f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:32.715067 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540540c-79dc-4b20-8cbc-7f085333d29f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:32.715345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540540c-79dc-4b20-8cbc-7f085333d29f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:32.716206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0540540c-79dc-4b20-8cbc-7f085333d29f None admin] RBAC: Adding query filter params (role.id=72bf8bd23d2c4b76ad32b7b35d6db412, scope.domain.id=default, user.id=1baa40dae37f4b3298a95bdc939c2f42) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:32.716519 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0540540c-79dc-4b20-8cbc-7f085333d29f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:32.718043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0540540c-79dc-4b20-8cbc-7f085333d29f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:32.727934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 12/21] 10.222.0.17 () {56 vars in 1322 bytes} [Tue Jul 26 04:45:32 2022] GET /identity/v3/role_assignments?user.id=1baa40dae37f4b3298a95bdc939c2f42&scope.domain.id=default&role.id=72bf8bd23d2c4b76ad32b7b35d6db412 => generated 233 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:45:35.283377 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a51355-60ff-483b-bec3-3b0394fa830a None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:35.284940 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a51355-60ff-483b-bec3-3b0394fa830a None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:35.285648 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a51355-60ff-483b-bec3-3b0394fa830a None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:35.288428 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 10/22] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:35.303660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f18d12-4eaf-4fcc-9764-3431057bc664 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:35.304879 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f18d12-4eaf-4fcc-9764-3431057bc664 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:35.305485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f18d12-4eaf-4fcc-9764-3431057bc664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:35.369270 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f3f18d12-4eaf-4fcc-9764-3431057bc664 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:35.371701 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f3f18d12-4eaf-4fcc-9764-3431057bc664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:35.433138 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:35.515971 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 13/23] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 04:45:35.580196 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a51355-60ff-483b-bec3-3b0394fa830a None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:35.670852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:35.688834 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf155a-fe28-439e-8f4d-8323b05f1780 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:35.689510 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf155a-fe28-439e-8f4d-8323b05f1780 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:35.690218 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf155a-fe28-439e-8f4d-8323b05f1780 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:35.691869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 14/24] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:35.710169 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cbc26-afd2-40f8-9e3f-0f5ab8c72590 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:35.710806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cbc26-afd2-40f8-9e3f-0f5ab8c72590 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:35.711288 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cbc26-afd2-40f8-9e3f-0f5ab8c72590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:35.749217 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-733cbc26-afd2-40f8-9e3f-0f5ab8c72590 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:35.750964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-733cbc26-afd2-40f8-9e3f-0f5ab8c72590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:35.823290 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a51355-60ff-483b-bec3-3b0394fa830a None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:35.892250 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 15/25] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 26 04:45:35.894951 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273f6a9-d105-443e-833e-2277865ad561 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:35.896428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273f6a9-d105-443e-833e-2277865ad561 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:35.897217 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273f6a9-d105-443e-833e-2277865ad561 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:35.898584 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb16ec8-c2b5-466d-bc73-1769f8215cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:35.900206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 16/26] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:35.901140 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb16ec8-c2b5-466d-bc73-1769f8215cad None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:35.901675 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb16ec8-c2b5-466d-bc73-1769f8215cad None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:35.902150 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb16ec8-c2b5-466d-bc73-1769f8215cad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:35.905721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cd19e-db64-4645-b926-31ee39123e75 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:35.906254 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cd19e-db64-4645-b926-31ee39123e75 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:35.907927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cd19e-db64-4645-b926-31ee39123e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:35.922628 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb16ec8-c2b5-466d-bc73-1769f8215cad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:45:35.933095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-0eb16ec8-c2b5-466d-bc73-1769f8215cad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:45:35.933095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:45:35.933095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:45:35.933095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:45:35.933095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:45:35.933095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:45:35.933095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:45:35.933095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:45:35.933095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:45:35.933095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:45:35.933095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:45:35.933095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:45:35.933095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:45:35.933095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:45:35.933095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:45:35.933095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:45:35.933962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:45:35.933962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:45:35.933962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:45:35.933962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:45:35.933962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:45:35.933962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:45:35.933962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:45:35.933962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:45:35.933962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:45:35.933962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:45:35.933962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:45:35.933962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:45:35.933962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:45:35.933962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:45:35.933962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:45:35.933962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:45:35.935112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:45:35.935112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:45:35.935112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:45:35.935112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:45:35.935112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:45:35.935112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:45:35.935112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:45:35.935112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:45:35.935112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:45:35.935112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:45:35.935112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:45:35.935112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:45:35.935112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 11/27] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:45:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:45:35.936010 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b22d03-c0bd-4299-896a-b88f780ae187 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:35.936010 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b22d03-c0bd-4299-896a-b88f780ae187 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:35.936121 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b22d03-c0bd-4299-896a-b88f780ae187 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:35.937627 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 12/28] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:35.942366 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b22d03-c0bd-4299-896a-b88f780ae187 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:35.946172 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4c2cd19e-db64-4645-b926-31ee39123e75 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:35.947503 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4c2cd19e-db64-4645-b926-31ee39123e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:36.038680 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 17/29] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:45:36.040061 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2cd19e-db64-4645-b926-31ee39123e75 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:36.062586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:36.065138 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:36.080636 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b22d03-c0bd-4299-896a-b88f780ae187 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:36.120310 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c9959-f867-48b2-91a9-e3ba8a6b0ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:36.121254 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c9959-f867-48b2-91a9-e3ba8a6b0ef6 None admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:36.121704 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c9959-f867-48b2-91a9-e3ba8a6b0ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:36.122107 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c9959-f867-48b2-91a9-e3ba8a6b0ef6 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:36.123018 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c9959-f867-48b2-91a9-e3ba8a6b0ef6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:36.124606 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c9959-f867-48b2-91a9-e3ba8a6b0ef6 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:36.164003 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2cd19e-db64-4645-b926-31ee39123e75 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:36.259149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 13/30] 10.222.0.17 () {60 vars in 1131 bytes} [Tue Jul 26 04:45:35 2022] POST /identity/v3/roles => generated 225 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:45:36.261239 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae070e9-9eec-42f4-9e02-4518cab76029 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:36.261767 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae070e9-9eec-42f4-9e02-4518cab76029 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:36.262245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae070e9-9eec-42f4-9e02-4518cab76029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:36.285066 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-4ae070e9-9eec-42f4-9e02-4518cab76029 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:36.287664 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-4ae070e9-9eec-42f4-9e02-4518cab76029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:36.321870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff1c29-0539-4f7c-8a00-2aeb273a686e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:36.322604 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff1c29-0539-4f7c-8a00-2aeb273a686e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:36.322814 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff1c29-0539-4f7c-8a00-2aeb273a686e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:36.323094 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff1c29-0539-4f7c-8a00-2aeb273a686e None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:36.323868 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff1c29-0539-4f7c-8a00-2aeb273a686e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:36.324333 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff1c29-0539-4f7c-8a00-2aeb273a686e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:36.326165 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff1c29-0539-4f7c-8a00-2aeb273a686e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:36.337418 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 18/31] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:45:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:36.338104 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff1c29-0539-4f7c-8a00-2aeb273a686e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:36.379637 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 14/32] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:45:36.380878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae070e9-9eec-42f4-9e02-4518cab76029 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:36.467750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:36.513064 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:36.519317 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff1c29-0539-4f7c-8a00-2aeb273a686e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:36.533886 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae070e9-9eec-42f4-9e02-4518cab76029 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:36.573137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de333b-5877-4bd1-8247-2bfe6e77f2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:36.573822 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de333b-5877-4bd1-8247-2bfe6e77f2d9 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:36.574108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de333b-5877-4bd1-8247-2bfe6e77f2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:36.574384 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de333b-5877-4bd1-8247-2bfe6e77f2d9 None admin] PATH_INFO: `/v3/users/1baa40dae37f4b3298a95bdc939c2f42` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:36.575665 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de333b-5877-4bd1-8247-2bfe6e77f2d9 None admin] RBAC: Authorizing `identity:get_user(user_id=1baa40dae37f4b3298a95bdc939c2f42)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:36.577526 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de333b-5877-4bd1-8247-2bfe6e77f2d9 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:36.579056 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 15/33] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:45:36 2022] GET /identity/v3/users/1baa40dae37f4b3298a95bdc939c2f42 => generated 247 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:36.579942 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de333b-5877-4bd1-8247-2bfe6e77f2d9 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:36.587191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94d696-4c12-4a98-97b2-71ea588913f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:36.588571 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94d696-4c12-4a98-97b2-71ea588913f9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:36.589314 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94d696-4c12-4a98-97b2-71ea588913f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:36.589948 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94d696-4c12-4a98-97b2-71ea588913f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:36.613734 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94d696-4c12-4a98-97b2-71ea588913f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:36.615884 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94d696-4c12-4a98-97b2-71ea588913f9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:36.633985 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-9f94d696-4c12-4a98-97b2-71ea588913f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:45:36.633985 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:45:36.633985 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:45:36.633985 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:45:36.633985 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:45:36.633985 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:45:36.633985 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:45:36.633985 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:45:36.633985 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:45:36.633985 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:45:36.633985 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:45:36.633985 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:45:36.633985 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:45:36.633985 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:45:36.633985 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:45:36.634855 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:45:36.634855 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:45:36.634855 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:45:36.634855 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:45:36.634855 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:45:36.634855 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:45:36.634855 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:45:36.634855 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:45:36.634855 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:45:36.634855 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:45:36.634855 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:45:36.634855 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:45:36.634855 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:45:36.634855 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:45:36.634855 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:45:36.634855 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:45:36.634855 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:45:36.635685 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:45:36.635685 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:45:36.635685 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:45:36.635685 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:45:36.637675 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 19/34] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:45:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:45:36.639318 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94d696-4c12-4a98-97b2-71ea588913f9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:36.694472 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de333b-5877-4bd1-8247-2bfe6e77f2d9 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:36.740144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add415a-9138-4781-8164-4d769264d455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:36.740845 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add415a-9138-4781-8164-4d769264d455 None admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:36.741137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add415a-9138-4781-8164-4d769264d455 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:36.741417 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add415a-9138-4781-8164-4d769264d455 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:36.742161 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add415a-9138-4781-8164-4d769264d455 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:36.743605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add415a-9138-4781-8164-4d769264d455 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:36.769900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 16/35] 10.222.0.17 () {60 vars in 1131 bytes} [Tue Jul 26 04:45:36 2022] POST /identity/v3/roles => generated 219 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:45:36.771928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add415a-9138-4781-8164-4d769264d455 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:36.787003 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94d696-4c12-4a98-97b2-71ea588913f9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:36.796571 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:36.828220 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:36.836147 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936d40a-b076-4724-9de8-bb4cff6c3fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:36.836823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936d40a-b076-4724-9de8-bb4cff6c3fca None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:36.838063 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936d40a-b076-4724-9de8-bb4cff6c3fca None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:36.838063 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936d40a-b076-4724-9de8-bb4cff6c3fca None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:36.838703 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936d40a-b076-4724-9de8-bb4cff6c3fca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:36.840365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936d40a-b076-4724-9de8-bb4cff6c3fca None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:36.842635 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 20/36] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:45:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:36.849427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32ae31-781f-428b-8b93-1b9ed83dd2e5 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:36.849744 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32ae31-781f-428b-8b93-1b9ed83dd2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:36.850109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32ae31-781f-428b-8b93-1b9ed83dd2e5 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:36.851887 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 21/37] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:36 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:36.852708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32ae31-781f-428b-8b93-1b9ed83dd2e5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:36.929690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add415a-9138-4781-8164-4d769264d455 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:36.970090 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0417c3-5de6-417a-b899-ab9c53389cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:36.971112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0417c3-5de6-417a-b899-ab9c53389cda None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:36.972026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0417c3-5de6-417a-b899-ab9c53389cda None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:36.972590 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0417c3-5de6-417a-b899-ab9c53389cda None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:36.973607 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0417c3-5de6-417a-b899-ab9c53389cda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:36.974203 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0417c3-5de6-417a-b899-ab9c53389cda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:36.975968 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0417c3-5de6-417a-b899-ab9c53389cda None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:36.977888 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32ae31-781f-428b-8b93-1b9ed83dd2e5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:36.989967 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 17/38] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:45:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:36.991651 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba80ab-d058-4325-a856-e57a37a63cdb None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:36.991861 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba80ab-d058-4325-a856-e57a37a63cdb None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:36.992233 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba80ab-d058-4325-a856-e57a37a63cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:37.013562 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-cdba80ab-d058-4325-a856-e57a37a63cdb None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:37.014870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-cdba80ab-d058-4325-a856-e57a37a63cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:37.024090 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515ea6c-05f8-450c-a982-7575c7c401a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:37.024794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515ea6c-05f8-450c-a982-7575c7c401a7 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:37.025134 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515ea6c-05f8-450c-a982-7575c7c401a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:37.025698 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515ea6c-05f8-450c-a982-7575c7c401a7 None admin] PATH_INFO: `/v3/domains/default/users/1baa40dae37f4b3298a95bdc939c2f42/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:37.027851 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6515ea6c-05f8-450c-a982-7575c7c401a7 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=1baa40dae37f4b3298a95bdc939c2f42, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:37.031757 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6515ea6c-05f8-450c-a982-7575c7c401a7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:37.090008 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 18/39] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:45:37.090685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba80ab-d058-4325-a856-e57a37a63cdb None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:37.102516 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:37.147422 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 22/40] 10.222.0.17 () {58 vars in 1361 bytes} [Tue Jul 26 04:45:36 2022] PUT /identity/v3/domains/default/users/1baa40dae37f4b3298a95bdc939c2f42/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:45:37.149350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c0256-99f6-4cc8-ac2e-49cec64d7634 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:37.149651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c0256-99f6-4cc8-ac2e-49cec64d7634 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:37.149963 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c0256-99f6-4cc8-ac2e-49cec64d7634 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:37.154688 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 23/41] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:37.155408 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c0256-99f6-4cc8-ac2e-49cec64d7634 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:37.226406 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba80ab-d058-4325-a856-e57a37a63cdb None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:37.231999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:37.263597 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9934e2-b7a7-4e23-8f0b-ea5b741b8108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:37.264181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9934e2-b7a7-4e23-8f0b-ea5b741b8108 None admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:37.264468 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9934e2-b7a7-4e23-8f0b-ea5b741b8108 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:37.264743 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9934e2-b7a7-4e23-8f0b-ea5b741b8108 None admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:37.265458 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9934e2-b7a7-4e23-8f0b-ea5b741b8108 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:37.267038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9934e2-b7a7-4e23-8f0b-ea5b741b8108 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:37.285478 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c0256-99f6-4cc8-ac2e-49cec64d7634 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:37.311579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 19/42] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jul 26 04:45:37 2022] POST /identity/v3/projects => generated 301 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:45:37.311579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02dbfa-8d75-469f-b0f8-00d42fdfd671 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:37.311579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02dbfa-8d75-469f-b0f8-00d42fdfd671 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:37.311579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02dbfa-8d75-469f-b0f8-00d42fdfd671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:37.333605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-3f02dbfa-8d75-469f-b0f8-00d42fdfd671 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:37.334856 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-3f02dbfa-8d75-469f-b0f8-00d42fdfd671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:37.339807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccac80f-aaf5-49fb-8b0f-d41e5e20a3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:37.341279 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccac80f-aaf5-49fb-8b0f-d41e5e20a3b3 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:37.341741 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccac80f-aaf5-49fb-8b0f-d41e5e20a3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:37.342207 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccac80f-aaf5-49fb-8b0f-d41e5e20a3b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:37.343125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccac80f-aaf5-49fb-8b0f-d41e5e20a3b3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:37.345366 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccac80f-aaf5-49fb-8b0f-d41e5e20a3b3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:37.371289 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 24/43] 10.222.0.17 () {60 vars in 1131 bytes} [Tue Jul 26 04:45:37 2022] POST /identity/v3/roles => generated 223 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:45:37.415976 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:37.433591 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 20/44] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:45:37.459706 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccac80f-aaf5-49fb-8b0f-d41e5e20a3b3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:37.587438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccac80f-aaf5-49fb-8b0f-d41e5e20a3b3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:37.605992 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:37.630340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0def56-f33a-46cc-8f4a-87f5ff49a5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:37.631015 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0def56-f33a-46cc-8f4a-87f5ff49a5cc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:37.631303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0def56-f33a-46cc-8f4a-87f5ff49a5cc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:37.631640 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0def56-f33a-46cc-8f4a-87f5ff49a5cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:37.632963 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0def56-f33a-46cc-8f4a-87f5ff49a5cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:37.634449 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0def56-f33a-46cc-8f4a-87f5ff49a5cc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:37.636276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 25/45] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:45:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:37.642692 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02dbfa-8d75-469f-b0f8-00d42fdfd671 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:37.752587 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02dbfa-8d75-469f-b0f8-00d42fdfd671 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:37.786614 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffece381-6dde-461e-9cb3-4fe2cc3b2661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:37.787248 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffece381-6dde-461e-9cb3-4fe2cc3b2661 None admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:37.787537 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffece381-6dde-461e-9cb3-4fe2cc3b2661 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:37.787844 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffece381-6dde-461e-9cb3-4fe2cc3b2661 None admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:37.788555 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffece381-6dde-461e-9cb3-4fe2cc3b2661 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:37.790036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffece381-6dde-461e-9cb3-4fe2cc3b2661 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:37.817459 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 21/46] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jul 26 04:45:37 2022] POST /identity/v3/projects => generated 312 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:45:39.235113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e282b-f49c-4d9a-ab23-a822fae8d9e3 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:39.235689 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e282b-f49c-4d9a-ab23-a822fae8d9e3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:39.235762 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e282b-f49c-4d9a-ab23-a822fae8d9e3 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:39.237142 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 26/47] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:39.242884 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e9bcd-8453-401b-b64d-2ff9566021ed None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:39.243315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e9bcd-8453-401b-b64d-2ff9566021ed None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:39.243434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e9bcd-8453-401b-b64d-2ff9566021ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:39.265107 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-999e9bcd-8453-401b-b64d-2ff9566021ed None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:39.266413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-999e9bcd-8453-401b-b64d-2ff9566021ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:39.298284 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:39.350901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 22/48] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:45:39.374358 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e282b-f49c-4d9a-ab23-a822fae8d9e3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:39.488180 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e282b-f49c-4d9a-ab23-a822fae8d9e3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:39.523095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921b588-c24c-47c0-a51f-c4a6c9945eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:39.523795 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921b588-c24c-47c0-a51f-c4a6c9945eaa None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:39.524093 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921b588-c24c-47c0-a51f-c4a6c9945eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:39.524362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921b588-c24c-47c0-a51f-c4a6c9945eaa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:39.535368 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6921b588-c24c-47c0-a51f-c4a6c9945eaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:45:39.540365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-6921b588-c24c-47c0-a51f-c4a6c9945eaa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:45:39.540365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:45:39.540365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:45:39.540365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:45:39.540365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:45:39.540365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:45:39.540365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:45:39.540365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:45:39.540365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:45:39.540365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:45:39.540365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:45:39.540365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:45:39.540365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:45:39.540365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:45:39.540365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:45:39.540365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:45:39.541276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:45:39.541276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:45:39.541276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:45:39.541276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:45:39.541276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:45:39.541276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:45:39.541276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:45:39.541276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:45:39.541276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:45:39.541276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:45:39.541276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:45:39.541276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:45:39.541276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:45:39.541276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:45:39.541276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:45:39.541276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:45:39.542026 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:45:39.542026 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:45:39.542026 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:45:39.542026 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:45:39.542026 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:45:39.542026 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:45:39.542026 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:45:39.542026 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:45:39.542026 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:45:39.542026 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:45:39.542026 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:45:39.542026 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:45:39.542026 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 27/49] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:45:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:45:39.547879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e9bcd-8453-401b-b64d-2ff9566021ed None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:39.660692 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e9bcd-8453-401b-b64d-2ff9566021ed None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:39.700186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e700a0-e01b-4325-99ce-253494e3ba04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:39.700816 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e700a0-e01b-4325-99ce-253494e3ba04 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:39.701111 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e700a0-e01b-4325-99ce-253494e3ba04 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:39.701387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e700a0-e01b-4325-99ce-253494e3ba04 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:39.702147 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e700a0-e01b-4325-99ce-253494e3ba04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:39.702459 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e700a0-e01b-4325-99ce-253494e3ba04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:39.703935 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e700a0-e01b-4325-99ce-253494e3ba04 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:39.717587 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 23/50] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:45:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:39.724187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921b588-c24c-47c0-a51f-c4a6c9945eaa None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:39.780789 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:39.815603 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d1eb7-5d90-4cfa-81b6-03989a27106a None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:39.815893 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d1eb7-5d90-4cfa-81b6-03989a27106a None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:39.816181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d1eb7-5d90-4cfa-81b6-03989a27106a None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:39.817562 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 24/51] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:39.823020 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04d9ea-96a2-43da-930e-83684f5fe870 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:39.823314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04d9ea-96a2-43da-930e-83684f5fe870 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:39.823584 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04d9ea-96a2-43da-930e-83684f5fe870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:39.842117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921b588-c24c-47c0-a51f-c4a6c9945eaa None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:39.842340 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-fe04d9ea-96a2-43da-930e-83684f5fe870 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:39.843563 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-fe04d9ea-96a2-43da-930e-83684f5fe870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:39.870537 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:39.880422 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66100f7a-646f-455d-8285-cd59655a89e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:39.881127 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66100f7a-646f-455d-8285-cd59655a89e7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:39.881406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66100f7a-646f-455d-8285-cd59655a89e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:39.881689 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66100f7a-646f-455d-8285-cd59655a89e7 None admin] PATH_INFO: `/v3/users/1baa40dae37f4b3298a95bdc939c2f42` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:39.882966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66100f7a-646f-455d-8285-cd59655a89e7 None admin] RBAC: Authorizing `identity:get_user(user_id=1baa40dae37f4b3298a95bdc939c2f42)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:39.884527 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66100f7a-646f-455d-8285-cd59655a89e7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:39.886293 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 28/52] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:45:39 2022] GET /identity/v3/users/1baa40dae37f4b3298a95bdc939c2f42 => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:39.892263 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66100f7a-646f-455d-8285-cd59655a89e7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:39.920788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 25/53] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 04:45:39.943857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe04d9ea-96a2-43da-930e-83684f5fe870 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:40.014167 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66100f7a-646f-455d-8285-cd59655a89e7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:40.030358 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:40.052680 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe04d9ea-96a2-43da-930e-83684f5fe870 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:40.053529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aac6a9-2335-4d62-b0a6-7a544bb87354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:40.054263 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aac6a9-2335-4d62-b0a6-7a544bb87354 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:40.054502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aac6a9-2335-4d62-b0a6-7a544bb87354 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:40.054792 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aac6a9-2335-4d62-b0a6-7a544bb87354 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:40.056000 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aac6a9-2335-4d62-b0a6-7a544bb87354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:40.057477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aac6a9-2335-4d62-b0a6-7a544bb87354 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:40.059140 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 29/54] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:45:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:45:40.065045 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aac6a9-2335-4d62-b0a6-7a544bb87354 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:40.096467 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f76bd3-cd1d-446f-b152-9ed480391f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:40.097142 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f76bd3-cd1d-446f-b152-9ed480391f62 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:40.097424 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f76bd3-cd1d-446f-b152-9ed480391f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:40.097696 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f76bd3-cd1d-446f-b152-9ed480391f62 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:40.098932 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f76bd3-cd1d-446f-b152-9ed480391f62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:40.100511 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f76bd3-cd1d-446f-b152-9ed480391f62 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:40.102133 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 26/55] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:45:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:45:40.108107 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f76bd3-cd1d-446f-b152-9ed480391f62 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:40.135704 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:40.169889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aac6a9-2335-4d62-b0a6-7a544bb87354 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:40.205229 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d172fd-f63c-4f83-bf11-06e6aee5602b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:40.205869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d172fd-f63c-4f83-bf11-06e6aee5602b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:40.206163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d172fd-f63c-4f83-bf11-06e6aee5602b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:40.206443 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d172fd-f63c-4f83-bf11-06e6aee5602b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:40.207250 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d172fd-f63c-4f83-bf11-06e6aee5602b None admin] RBAC: Adding query filter params (role.id=72bf8bd23d2c4b76ad32b7b35d6db412, scope.domain.id=default, user.id=1baa40dae37f4b3298a95bdc939c2f42) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:40.207572 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d172fd-f63c-4f83-bf11-06e6aee5602b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:40.209055 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d172fd-f63c-4f83-bf11-06e6aee5602b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:40.213322 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f76bd3-cd1d-446f-b152-9ed480391f62 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:40.214951 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 30/56] 10.222.0.17 () {56 vars in 1322 bytes} [Tue Jul 26 04:45:40 2022] GET /identity/v3/role_assignments?user.id=1baa40dae37f4b3298a95bdc939c2f42&scope.domain.id=default&role.id=72bf8bd23d2c4b76ad32b7b35d6db412 => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:45:40.251667 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba0193c-331c-4e02-9f4d-0e84a853fb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:40.252323 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0193c-331c-4e02-9f4d-0e84a853fb03 None admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:40.252613 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0193c-331c-4e02-9f4d-0e84a853fb03 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:40.252896 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0193c-331c-4e02-9f4d-0e84a853fb03 None admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:40.253628 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba0193c-331c-4e02-9f4d-0e84a853fb03 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:40.255155 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba0193c-331c-4e02-9f4d-0e84a853fb03 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:40.287491 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 27/57] 10.222.0.17 () {60 vars in 1140 bytes} [Tue Jul 26 04:45:40 2022] POST /identity/v3/projects => generated 298 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:45:42.292345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5fcc1-cca4-4327-8a68-2a1913a581cb None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:42.292888 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5fcc1-cca4-4327-8a68-2a1913a581cb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:42.292888 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5fcc1-cca4-4327-8a68-2a1913a581cb None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:42.294303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 31/58] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:42.299841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0141ec6-4229-40c7-930f-8005b068a7d1 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:42.300324 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0141ec6-4229-40c7-930f-8005b068a7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:42.300408 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0141ec6-4229-40c7-930f-8005b068a7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:42.320503 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-e0141ec6-4229-40c7-930f-8005b068a7d1 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:42.321714 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-e0141ec6-4229-40c7-930f-8005b068a7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:42.407365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 28/59] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:45:42.431104 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5fcc1-cca4-4327-8a68-2a1913a581cb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:42.483961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:42.544130 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5fcc1-cca4-4327-8a68-2a1913a581cb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:42.580284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96699b-1e45-48b9-9244-62bba33717ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:42.580941 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96699b-1e45-48b9-9244-62bba33717ed None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:42.581219 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96699b-1e45-48b9-9244-62bba33717ed None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:42.581494 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96699b-1e45-48b9-9244-62bba33717ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:42.582780 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e96699b-1e45-48b9-9244-62bba33717ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:42.584300 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e96699b-1e45-48b9-9244-62bba33717ed None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:42.585923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 32/60] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:45:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:42.591818 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0141ec6-4229-40c7-930f-8005b068a7d1 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:42.602748 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:42.704772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0141ec6-4229-40c7-930f-8005b068a7d1 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:42.743160 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376fbbe-f93f-4b3a-be8e-0e060d951a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:42.743842 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376fbbe-f93f-4b3a-be8e-0e060d951a84 None admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:42.744126 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376fbbe-f93f-4b3a-be8e-0e060d951a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:42.744464 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376fbbe-f93f-4b3a-be8e-0e060d951a84 None admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:42.745125 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376fbbe-f93f-4b3a-be8e-0e060d951a84 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:42.746627 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376fbbe-f93f-4b3a-be8e-0e060d951a84 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:42.776987 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 29/61] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jul 26 04:45:42 2022] POST /identity/v3/users => generated 275 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:45:45.873714 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815e017-5fae-417b-a26e-224a8efd37d7 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:45.874231 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815e017-5fae-417b-a26e-224a8efd37d7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:45.874305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815e017-5fae-417b-a26e-224a8efd37d7 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:45.875761 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 33/62] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:45.881321 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13328a67-4d20-4d0a-9152-02b1ee8c9320 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:45.882217 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13328a67-4d20-4d0a-9152-02b1ee8c9320 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:45.882693 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13328a67-4d20-4d0a-9152-02b1ee8c9320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:45.903179 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-13328a67-4d20-4d0a-9152-02b1ee8c9320 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:45.904913 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-13328a67-4d20-4d0a-9152-02b1ee8c9320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:45.998343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:46.008356 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 30/63] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:45:46.060053 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f1611b-d154-4a63-832e-4ae1a289320a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:46.060675 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f1611b-d154-4a63-832e-4ae1a289320a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:46.061145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f1611b-d154-4a63-832e-4ae1a289320a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:46.063022 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 34/64] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:46.068392 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf0b96-d0d2-4210-9c06-0e9232bd422c None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:46.068949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf0b96-d0d2-4210-9c06-0e9232bd422c None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:46.069412 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf0b96-d0d2-4210-9c06-0e9232bd422c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:46.071972 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f1611b-d154-4a63-832e-4ae1a289320a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:46.097033 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-7cbf0b96-d0d2-4210-9c06-0e9232bd422c None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:46.098907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-7cbf0b96-d0d2-4210-9c06-0e9232bd422c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:46.224255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 31/65] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:45:46.226899 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10be438-cfbb-4ffe-89f2-4dd4644c7beb None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:46.227654 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10be438-cfbb-4ffe-89f2-4dd4644c7beb None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:46.228343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10be438-cfbb-4ffe-89f2-4dd4644c7beb None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:46.231021 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 32/66] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:46.240050 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f1611b-d154-4a63-832e-4ae1a289320a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:46.250476 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e66f8-9571-438b-bdb8-d618934e7bcf None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:46.250762 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e66f8-9571-438b-bdb8-d618934e7bcf None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:46.253233 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e66f8-9571-438b-bdb8-d618934e7bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:46.261740 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:46.282646 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-d50e66f8-9571-438b-bdb8-d618934e7bcf None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:46.283899 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-d50e66f8-9571-438b-bdb8-d618934e7bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:46.293114 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9db3c-75e3-4da4-8f0e-d7834f4cf162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:46.293944 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9db3c-75e3-4da4-8f0e-d7834f4cf162 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:46.294234 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9db3c-75e3-4da4-8f0e-d7834f4cf162 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:46.294514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9db3c-75e3-4da4-8f0e-d7834f4cf162 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:46.308929 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9db3c-75e3-4da4-8f0e-d7834f4cf162 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:45:46.315568 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d6c9db3c-75e3-4da4-8f0e-d7834f4cf162 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:45:46.315568 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:45:46.315568 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:45:46.315568 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:45:46.315568 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:45:46.315568 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:45:46.315568 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:45:46.315568 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:45:46.315568 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:45:46.315568 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:45:46.315568 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:45:46.315568 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:45:46.315568 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:45:46.315568 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:45:46.315568 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:45:46.315568 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:45:46.316622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:45:46.316622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:45:46.316622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:45:46.316622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:45:46.316622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:45:46.316622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:45:46.316622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:45:46.316622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:45:46.316622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:45:46.316622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:45:46.316622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:45:46.316622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:45:46.316622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:45:46.316622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:45:46.316622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:45:46.316622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:45:46.317499 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:45:46.317499 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:45:46.317499 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:45:46.317499 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:45:46.317499 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:45:46.317499 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:45:46.317499 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:45:46.317499 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:45:46.317499 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:45:46.317499 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:45:46.317499 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:45:46.317499 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:45:46.319861 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 35/67] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:45:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:45:46.326327 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9db3c-75e3-4da4-8f0e-d7834f4cf162 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:46.386117 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 33/68] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 04:45:46.386117 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e66f8-9571-438b-bdb8-d618934e7bcf None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:46.442736 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:46.471831 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9db3c-75e3-4da4-8f0e-d7834f4cf162 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:46.504737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e66f8-9571-438b-bdb8-d618934e7bcf None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:46.507864 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f97c51-b4af-4e8a-b996-55f51b1fadbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:46.508839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f97c51-b4af-4e8a-b996-55f51b1fadbf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:46.509335 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f97c51-b4af-4e8a-b996-55f51b1fadbf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:46.509817 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f97c51-b4af-4e8a-b996-55f51b1fadbf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:46.521930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f97c51-b4af-4e8a-b996-55f51b1fadbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:45:46.528109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-52f97c51-b4af-4e8a-b996-55f51b1fadbf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:45:46.528109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:45:46.528109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:45:46.528109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:45:46.528109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:45:46.528109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:45:46.528109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:45:46.528109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:45:46.528109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:45:46.528109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:45:46.528109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:45:46.528109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:45:46.528109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:45:46.528109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:45:46.528109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:45:46.528109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:45:46.528961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:45:46.528961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:45:46.528961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:45:46.528961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:45:46.528961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:45:46.528961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:45:46.528961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:45:46.528961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:45:46.528961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:45:46.528961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:45:46.528961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:45:46.528961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:45:46.528961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:45:46.528961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:45:46.528961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:45:46.528961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:45:46.529732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:45:46.529732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:45:46.529732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:45:46.529732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:45:46.529732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:45:46.529732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:45:46.529732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:45:46.529732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:45:46.529732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:45:46.529732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:45:46.529732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:45:46.529732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:45:46.532392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 36/69] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:45:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:45:46.533538 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f97c51-b4af-4e8a-b996-55f51b1fadbf None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:46.545988 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f6597-650a-4202-86a4-e888c525710f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:46.547037 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f6597-650a-4202-86a4-e888c525710f None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:46.547498 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f6597-650a-4202-86a4-e888c525710f None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:46.547966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f6597-650a-4202-86a4-e888c525710f None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:46.548867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f6597-650a-4202-86a4-e888c525710f None admin] RBAC: Adding query filter params (name=member) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:46.549342 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f6597-650a-4202-86a4-e888c525710f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:46.551126 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f6597-650a-4202-86a4-e888c525710f None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:46.561138 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 34/70] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:45:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:46.562386 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f6597-650a-4202-86a4-e888c525710f None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:46.570750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:46.646296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f97c51-b4af-4e8a-b996-55f51b1fadbf None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:46.674769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f6597-650a-4202-86a4-e888c525710f None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:46.683657 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed0e555-7bcf-4aa8-a8cd-36e71a0e8dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:46.684428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0e555-7bcf-4aa8-a8cd-36e71a0e8dad None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:46.684710 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0e555-7bcf-4aa8-a8cd-36e71a0e8dad None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:46.684986 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0e555-7bcf-4aa8-a8cd-36e71a0e8dad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:46.691497 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:46.697866 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed0e555-7bcf-4aa8-a8cd-36e71a0e8dad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:45:46.703298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-0ed0e555-7bcf-4aa8-a8cd-36e71a0e8dad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 04:45:46.703298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:45:46.703298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:45:46.703298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:45:46.703298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:45:46.703298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:45:46.703298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:45:46.703298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:45:46.703298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:45:46.703298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:45:46.703298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:45:46.703298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:45:46.703298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:45:46.703298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:45:46.703298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:45:46.703298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:45:46.704166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:45:46.704166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:45:46.704166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:45:46.704166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:45:46.704166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:45:46.704166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:45:46.704166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:45:46.704166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:45:46.704166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:45:46.704166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:45:46.704166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:45:46.704166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:45:46.704166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:45:46.704166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:45:46.704166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:45:46.704166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:45:46.704916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:45:46.704916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:45:46.704916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:45:46.704916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:45:46.704916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:45:46.704916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:45:46.704916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:45:46.704916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:45:46.704916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:45:46.704916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:45:46.704916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 04:45:46.704916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:45:46.704916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 37/71] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jul 26 04:45:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:45:46.705535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed0e555-7bcf-4aa8-a8cd-36e71a0e8dad None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:46.716578 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89db758-d1d5-448f-b3c4-d807aca253cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:46.717220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89db758-d1d5-448f-b3c4-d807aca253cd None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:46.717575 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89db758-d1d5-448f-b3c4-d807aca253cd None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:46.717869 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89db758-d1d5-448f-b3c4-d807aca253cd None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:46.718649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89db758-d1d5-448f-b3c4-d807aca253cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:46.718964 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89db758-d1d5-448f-b3c4-d807aca253cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:46.720410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89db758-d1d5-448f-b3c4-d807aca253cd None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:46.729458 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 35/72] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:45:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:46.731713 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fdbec7-6e79-4511-b90e-c0e115b2cd7c None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:46.731713 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fdbec7-6e79-4511-b90e-c0e115b2cd7c None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:46.731713 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fdbec7-6e79-4511-b90e-c0e115b2cd7c None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:46.732518 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 36/73] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:46.733204 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fdbec7-6e79-4511-b90e-c0e115b2cd7c None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:46.806160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:46.813205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed0e555-7bcf-4aa8-a8cd-36e71a0e8dad None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:46.839162 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fdbec7-6e79-4511-b90e-c0e115b2cd7c None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:46.848323 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6e9de6-3667-4be9-aa6f-227429d8cd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:46.848984 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e9de6-3667-4be9-aa6f-227429d8cd37 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:46.849269 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e9de6-3667-4be9-aa6f-227429d8cd37 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:46.849546 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e9de6-3667-4be9-aa6f-227429d8cd37 None admin] PATH_INFO: `/v3/users/7b1c7d019a3b49b79a0a9a3a4aea88f9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:46.865381 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6e9de6-3667-4be9-aa6f-227429d8cd37 None admin] RBAC: Authorizing `identity:get_user(user_id=7b1c7d019a3b49b79a0a9a3a4aea88f9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:46.866880 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6e9de6-3667-4be9-aa6f-227429d8cd37 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:46.868629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 38/74] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:45:46 2022] GET /identity/v3/users/7b1c7d019a3b49b79a0a9a3a4aea88f9 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:46.869406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6e9de6-3667-4be9-aa6f-227429d8cd37 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:46.878855 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0bb0e0-0344-4a8e-8490-b394216906f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:46.879487 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0bb0e0-0344-4a8e-8490-b394216906f6 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:46.880174 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0bb0e0-0344-4a8e-8490-b394216906f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:46.880520 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0bb0e0-0344-4a8e-8490-b394216906f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:46.881487 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0bb0e0-0344-4a8e-8490-b394216906f6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:46.882044 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0bb0e0-0344-4a8e-8490-b394216906f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:46.883466 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0bb0e0-0344-4a8e-8490-b394216906f6 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:46.892591 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 37/75] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:45:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:46.896921 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5462f9b-4438-4da5-8a7d-b918e6307d6d None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:46.897297 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5462f9b-4438-4da5-8a7d-b918e6307d6d None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:46.897580 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5462f9b-4438-4da5-8a7d-b918e6307d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:46.915062 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-f5462f9b-4438-4da5-8a7d-b918e6307d6d None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:46.916313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-f5462f9b-4438-4da5-8a7d-b918e6307d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:46.951361 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:46.968962 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6e9de6-3667-4be9-aa6f-227429d8cd37 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:47.001870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 38/76] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:45:47.005032 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6bb97e-2f1a-41cc-adf6-24dfd8245a02 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:47.005318 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6bb97e-2f1a-41cc-adf6-24dfd8245a02 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:47.005595 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6bb97e-2f1a-41cc-adf6-24dfd8245a02 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:47.006956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 39/77] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:47.011744 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6bb97e-2f1a-41cc-adf6-24dfd8245a02 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:47.013099 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03629425-c1a7-4470-b774-257936afb6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:47.013871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03629425-c1a7-4470-b774-257936afb6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:47.014138 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03629425-c1a7-4470-b774-257936afb6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:47.014420 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03629425-c1a7-4470-b774-257936afb6a9 None admin] PATH_INFO: `/v3/users/1baa40dae37f4b3298a95bdc939c2f42` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:47.015808 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03629425-c1a7-4470-b774-257936afb6a9 None admin] RBAC: Authorizing `identity:get_user(user_id=1baa40dae37f4b3298a95bdc939c2f42)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:47.017306 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03629425-c1a7-4470-b774-257936afb6a9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:47.019147 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 39/78] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:45:46 2022] GET /identity/v3/users/1baa40dae37f4b3298a95bdc939c2f42 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:47.019740 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03629425-c1a7-4470-b774-257936afb6a9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:47.042834 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:47.130990 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03629425-c1a7-4470-b774-257936afb6a9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:47.133378 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6bb97e-2f1a-41cc-adf6-24dfd8245a02 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:47.168237 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40116f57-56a9-40aa-8c0b-4a8b693730f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:47.168549 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c078dd-6c8c-4ebc-b58c-2e6f284b5f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:47.168901 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40116f57-56a9-40aa-8c0b-4a8b693730f7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:47.169169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c078dd-6c8c-4ebc-b58c-2e6f284b5f1c None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:47.169259 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40116f57-56a9-40aa-8c0b-4a8b693730f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:47.169441 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40116f57-56a9-40aa-8c0b-4a8b693730f7 None admin] PATH_INFO: `/v3/users/7b1c7d019a3b49b79a0a9a3a4aea88f9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:47.169525 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c078dd-6c8c-4ebc-b58c-2e6f284b5f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:47.169718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c078dd-6c8c-4ebc-b58c-2e6f284b5f1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:47.170671 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40116f57-56a9-40aa-8c0b-4a8b693730f7 None admin] RBAC: Authorizing `identity:get_user(user_id=7b1c7d019a3b49b79a0a9a3a4aea88f9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:47.172183 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40116f57-56a9-40aa-8c0b-4a8b693730f7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:47.173810 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 40/79] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:45:47 2022] GET /identity/v3/users/7b1c7d019a3b49b79a0a9a3a4aea88f9 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:47.175642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6a788-a3d8-42e2-8784-9db502fe0bac None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:47.175642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6a788-a3d8-42e2-8784-9db502fe0bac None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:47.175642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6a788-a3d8-42e2-8784-9db502fe0bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:47.196692 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7ac6a788-a3d8-42e2-8784-9db502fe0bac None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:47.198262 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7ac6a788-a3d8-42e2-8784-9db502fe0bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:47.214177 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:47.243824 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c078dd-6c8c-4ebc-b58c-2e6f284b5f1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:47.245333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c078dd-6c8c-4ebc-b58c-2e6f284b5f1c None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:47.265702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 41/80] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:45:47.266585 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6a788-a3d8-42e2-8784-9db502fe0bac None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:47.286169 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:47.370625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 40/81] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:45:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:45:47.370625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c078dd-6c8c-4ebc-b58c-2e6f284b5f1c None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:47.380872 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6a788-a3d8-42e2-8784-9db502fe0bac None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:47.420324 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c0e1e3-a8b2-4b75-88ec-01cc9ba72ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:47.421189 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0e1e3-a8b2-4b75-88ec-01cc9ba72ebd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:47.421287 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0e1e3-a8b2-4b75-88ec-01cc9ba72ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:47.421535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0e1e3-a8b2-4b75-88ec-01cc9ba72ebd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:47.422828 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c0e1e3-a8b2-4b75-88ec-01cc9ba72ebd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:47.424343 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c0e1e3-a8b2-4b75-88ec-01cc9ba72ebd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:47.426003 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 42/82] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:45:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:47.426783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c0e1e3-a8b2-4b75-88ec-01cc9ba72ebd None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:47.445674 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:47.488946 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c078dd-6c8c-4ebc-b58c-2e6f284b5f1c None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:47.524031 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273d4e1-bf03-4c9e-b629-8910f4e1628b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:47.524664 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273d4e1-bf03-4c9e-b629-8910f4e1628b None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:47.524948 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273d4e1-bf03-4c9e-b629-8910f4e1628b None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:47.525226 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273d4e1-bf03-4c9e-b629-8910f4e1628b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:47.527069 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:47.544077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c0e1e3-a8b2-4b75-88ec-01cc9ba72ebd None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:47.579408 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551cb511-1f62-4754-bfc0-e9af9d9985c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:47.580111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cb511-1f62-4754-bfc0-e9af9d9985c8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:47.580352 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cb511-1f62-4754-bfc0-e9af9d9985c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:47.580625 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cb511-1f62-4754-bfc0-e9af9d9985c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:47.597548 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273d4e1-bf03-4c9e-b629-8910f4e1628b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:47.598952 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273d4e1-bf03-4c9e-b629-8910f4e1628b None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:47.650584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551cb511-1f62-4754-bfc0-e9af9d9985c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:47.652042 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551cb511-1f62-4754-bfc0-e9af9d9985c8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:47.670832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:47.721335 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 41/83] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:45:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:45:47.721335 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273d4e1-bf03-4c9e-b629-8910f4e1628b None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:47.753567 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:47.770271 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 43/84] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:45:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:45:47.771145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551cb511-1f62-4754-bfc0-e9af9d9985c8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:47.837770 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273d4e1-bf03-4c9e-b629-8910f4e1628b None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:47.876650 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551cb511-1f62-4754-bfc0-e9af9d9985c8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:47.878099 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd368c2-ce2c-4a0b-8ba2-1e979a1e1090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:47.878949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd368c2-ce2c-4a0b-8ba2-1e979a1e1090 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:47.879908 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd368c2-ce2c-4a0b-8ba2-1e979a1e1090 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:47.880210 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd368c2-ce2c-4a0b-8ba2-1e979a1e1090 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:47.891957 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd368c2-ce2c-4a0b-8ba2-1e979a1e1090 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:45:47.897047 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-dfd368c2-ce2c-4a0b-8ba2-1e979a1e1090 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:45:47.897047 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:45:47.897047 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:45:47.897047 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:45:47.897047 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:45:47.897047 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:45:47.897047 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:45:47.897047 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:45:47.897047 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:45:47.897047 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:45:47.897047 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:45:47.897047 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:45:47.897047 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:45:47.897047 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:45:47.897047 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:45:47.897047 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:45:47.897996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:45:47.897996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:45:47.897996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:45:47.897996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:45:47.897996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:45:47.897996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:45:47.897996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:45:47.897996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:45:47.897996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:45:47.897996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:45:47.897996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:45:47.897996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:45:47.897996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:45:47.897996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:45:47.897996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:45:47.897996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:45:47.904149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:45:47.904149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:45:47.904149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:45:47.904149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:45:47.904149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:45:47.904149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:45:47.904149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:45:47.904149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:45:47.904149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:45:47.904149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:45:47.904149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:45:47.904149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:45:47.904149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 42/85] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:45:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:45:47.904149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd368c2-ce2c-4a0b-8ba2-1e979a1e1090 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:47.913425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8553f-0a2b-4ef3-858c-90e3f0717f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:47.914096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8553f-0a2b-4ef3-858c-90e3f0717f01 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:47.914372 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8553f-0a2b-4ef3-858c-90e3f0717f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:47.914644 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8553f-0a2b-4ef3-858c-90e3f0717f01 None admin] PATH_INFO: `/v3/projects/8be3583b9359412ba4129e556beef77c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:47.915847 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8553f-0a2b-4ef3-858c-90e3f0717f01 None admin] RBAC: Authorizing `identity:get_project(project_id=8be3583b9359412ba4129e556beef77c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:47.917348 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8553f-0a2b-4ef3-858c-90e3f0717f01 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:47.919046 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 44/86] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:45:47 2022] GET /identity/v3/projects/8be3583b9359412ba4129e556beef77c => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:47.919837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8553f-0a2b-4ef3-858c-90e3f0717f01 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:47.941276 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:47.979887 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:48.013817 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd368c2-ce2c-4a0b-8ba2-1e979a1e1090 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:48.026735 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8553f-0a2b-4ef3-858c-90e3f0717f01 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:48.049990 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752aa6a-e934-446e-ae30-ff173fe2f344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:48.050182 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752aa6a-e934-446e-ae30-ff173fe2f344 None admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:48.050343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752aa6a-e934-446e-ae30-ff173fe2f344 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:48.050710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752aa6a-e934-446e-ae30-ff173fe2f344 None admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:48.051331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5752aa6a-e934-446e-ae30-ff173fe2f344 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:48.052941 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5752aa6a-e934-446e-ae30-ff173fe2f344 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:48.061505 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fadcc-3b83-431a-b061-80478f31c057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:48.062298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fadcc-3b83-431a-b061-80478f31c057 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:48.062678 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fadcc-3b83-431a-b061-80478f31c057 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:48.062865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fadcc-3b83-431a-b061-80478f31c057 None admin] PATH_INFO: `/v3/projects/8be3583b9359412ba4129e556beef77c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:48.064117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450fadcc-3b83-431a-b061-80478f31c057 None admin] RBAC: Authorizing `identity:get_project(project_id=8be3583b9359412ba4129e556beef77c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:48.065583 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450fadcc-3b83-431a-b061-80478f31c057 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:48.067301 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 45/87] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:45:47 2022] GET /identity/v3/projects/8be3583b9359412ba4129e556beef77c => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:48.068112 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fadcc-3b83-431a-b061-80478f31c057 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:48.078430 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 43/88] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jul 26 04:45:47 2022] POST /identity/v3/users => generated 289 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:45:48.079271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752aa6a-e934-446e-ae30-ff173fe2f344 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:48.181193 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:48.197454 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fadcc-3b83-431a-b061-80478f31c057 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:48.202213 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752aa6a-e934-446e-ae30-ff173fe2f344 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:48.223196 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:48.239315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c2323-78f3-4c27-a8dc-1fa47ea1d75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:48.240056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c2323-78f3-4c27-a8dc-1fa47ea1d75d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:48.240340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c2323-78f3-4c27-a8dc-1fa47ea1d75d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:48.240614 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c2323-78f3-4c27-a8dc-1fa47ea1d75d None admin] PATH_INFO: `/v3/projects/8be3583b9359412ba4129e556beef77c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:48.242218 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c2323-78f3-4c27-a8dc-1fa47ea1d75d None admin] RBAC: Authorizing `identity:get_project(project_id=8be3583b9359412ba4129e556beef77c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:48.243272 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c2323-78f3-4c27-a8dc-1fa47ea1d75d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:48.245249 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 46/89] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:45:48 2022] GET /identity/v3/projects/8be3583b9359412ba4129e556beef77c => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:48.250277 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c2323-78f3-4c27-a8dc-1fa47ea1d75d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:48.252031 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b72a9e8-63c6-4312-9172-24f3634309bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:48.252717 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72a9e8-63c6-4312-9172-24f3634309bf None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:48.253006 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72a9e8-63c6-4312-9172-24f3634309bf None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:48.254087 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72a9e8-63c6-4312-9172-24f3634309bf None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:48.254087 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b72a9e8-63c6-4312-9172-24f3634309bf None admin] RBAC: Adding query filter params (name=member) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:48.254393 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b72a9e8-63c6-4312-9172-24f3634309bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:48.256396 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b72a9e8-63c6-4312-9172-24f3634309bf None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:48.266269 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 44/90] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:45:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:48.267153 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b72a9e8-63c6-4312-9172-24f3634309bf None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:48.373023 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c2323-78f3-4c27-a8dc-1fa47ea1d75d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:48.409348 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b72a9e8-63c6-4312-9172-24f3634309bf None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:48.423032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb130401-76a3-429e-83fb-9ae6d4550781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:48.424392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb130401-76a3-429e-83fb-9ae6d4550781 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:48.424561 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb130401-76a3-429e-83fb-9ae6d4550781 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:48.424833 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb130401-76a3-429e-83fb-9ae6d4550781 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:48.426231 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb130401-76a3-429e-83fb-9ae6d4550781 None admin] RBAC: Adding query filter params (role.id=7519356859fb40508c25bec8161207fd, scope.project.id=8be3583b9359412ba4129e556beef77c, user.id=7b1c7d019a3b49b79a0a9a3a4aea88f9) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:48.426231 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb130401-76a3-429e-83fb-9ae6d4550781 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:48.427451 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb130401-76a3-429e-83fb-9ae6d4550781 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:48.434824 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 47/91] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:45:48 2022] GET /identity/v3/role_assignments?user.id=7b1c7d019a3b49b79a0a9a3a4aea88f9&scope.project.id=8be3583b9359412ba4129e556beef77c&role.id=7519356859fb40508c25bec8161207fd => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:48.435848 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb130401-76a3-429e-83fb-9ae6d4550781 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:48.457352 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa03da6-dbb1-4da4-9fb0-c35123720da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:48.458185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa03da6-dbb1-4da4-9fb0-c35123720da4 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:48.458489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa03da6-dbb1-4da4-9fb0-c35123720da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:48.458770 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa03da6-dbb1-4da4-9fb0-c35123720da4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:48.459800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa03da6-dbb1-4da4-9fb0-c35123720da4 None admin] RBAC: Adding query filter params (role.id=72bf8bd23d2c4b76ad32b7b35d6db412, scope.project.id=8be3583b9359412ba4129e556beef77c, user.id=1baa40dae37f4b3298a95bdc939c2f42) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:48.460122 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa03da6-dbb1-4da4-9fb0-c35123720da4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:48.461743 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa03da6-dbb1-4da4-9fb0-c35123720da4 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:48.471736 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 45/92] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:45:48 2022] GET /identity/v3/role_assignments?user.id=1baa40dae37f4b3298a95bdc939c2f42&scope.project.id=8be3583b9359412ba4129e556beef77c&role.id=72bf8bd23d2c4b76ad32b7b35d6db412 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:48.472928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa03da6-dbb1-4da4-9fb0-c35123720da4 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:48.509074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:48.538159 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:48.644192 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb130401-76a3-429e-83fb-9ae6d4550781 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:48.653957 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa03da6-dbb1-4da4-9fb0-c35123720da4 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:48.698135 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af31d12-111d-442c-8228-0de54d4f1e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:48.699570 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af31d12-111d-442c-8228-0de54d4f1e20 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:48.699920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af31d12-111d-442c-8228-0de54d4f1e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:48.700205 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af31d12-111d-442c-8228-0de54d4f1e20 None admin] PATH_INFO: `/v3/users/7b1c7d019a3b49b79a0a9a3a4aea88f9` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:48.701833 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af31d12-111d-442c-8228-0de54d4f1e20 None admin] RBAC: Authorizing `identity:get_user(user_id=7b1c7d019a3b49b79a0a9a3a4aea88f9)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:48.703427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af31d12-111d-442c-8228-0de54d4f1e20 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:48.705752 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 46/93] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:45:48 2022] GET /identity/v3/users/7b1c7d019a3b49b79a0a9a3a4aea88f9 => generated 275 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:48.712551 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af31d12-111d-442c-8228-0de54d4f1e20 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:48.737748 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e43bd0-874d-445f-8085-2a7190a03120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:48.738680 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e43bd0-874d-445f-8085-2a7190a03120 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:48.738970 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e43bd0-874d-445f-8085-2a7190a03120 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:48.740168 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e43bd0-874d-445f-8085-2a7190a03120 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:48.741019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e43bd0-874d-445f-8085-2a7190a03120 None admin] RBAC: Adding query filter params (role.id=9223a9bb40484faca3a7bcfeb41a2215, scope.project.id=8be3583b9359412ba4129e556beef77c, user.id=7b1c7d019a3b49b79a0a9a3a4aea88f9) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:48.741331 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e43bd0-874d-445f-8085-2a7190a03120 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:48.742828 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e43bd0-874d-445f-8085-2a7190a03120 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:48.758397 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 48/94] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:45:48 2022] GET /identity/v3/role_assignments?user.id=7b1c7d019a3b49b79a0a9a3a4aea88f9&scope.project.id=8be3583b9359412ba4129e556beef77c&role.id=9223a9bb40484faca3a7bcfeb41a2215 => generated 259 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:48.838605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:48.873744 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af31d12-111d-442c-8228-0de54d4f1e20 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:48.927048 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc6979-75f4-4b13-b185-ba8f9f36cff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:48.927699 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc6979-75f4-4b13-b185-ba8f9f36cff0 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:48.927985 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc6979-75f4-4b13-b185-ba8f9f36cff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:48.928264 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc6979-75f4-4b13-b185-ba8f9f36cff0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:49.023441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dc6979-75f4-4b13-b185-ba8f9f36cff0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:49.026291 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dc6979-75f4-4b13-b185-ba8f9f36cff0 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:49.154285 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:49.181173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 47/95] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:45:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:45:49.187170 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e43bd0-874d-445f-8085-2a7190a03120 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:49.334605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e43bd0-874d-445f-8085-2a7190a03120 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:49.360215 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:49.386577 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f805b0a-439a-48f1-8ba8-f89d46a70809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:49.386577 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f805b0a-439a-48f1-8ba8-f89d46a70809 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:49.386577 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f805b0a-439a-48f1-8ba8-f89d46a70809 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:49.386577 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f805b0a-439a-48f1-8ba8-f89d46a70809 None admin] PATH_INFO: `/v3/projects/6404c767e4bf43c8aa64c294fd5de203` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:49.387784 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f805b0a-439a-48f1-8ba8-f89d46a70809 None admin] RBAC: Authorizing `identity:get_project(project_id=6404c767e4bf43c8aa64c294fd5de203)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:49.389563 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f805b0a-439a-48f1-8ba8-f89d46a70809 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:49.391471 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 49/96] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:45:49 2022] GET /identity/v3/projects/6404c767e4bf43c8aa64c294fd5de203 => generated 312 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:49.397575 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc6979-75f4-4b13-b185-ba8f9f36cff0 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:49.561688 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc6979-75f4-4b13-b185-ba8f9f36cff0 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:49.633833 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1cd724-86c7-41c8-81f7-12d71ee27a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:49.634876 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cd724-86c7-41c8-81f7-12d71ee27a63 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:49.635360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cd724-86c7-41c8-81f7-12d71ee27a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:49.635864 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cd724-86c7-41c8-81f7-12d71ee27a63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:49.637371 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1cd724-86c7-41c8-81f7-12d71ee27a63 None admin] RBAC: Adding query filter params (role.id=7519356859fb40508c25bec8161207fd, scope.project.id=6404c767e4bf43c8aa64c294fd5de203, user.id=7b1c7d019a3b49b79a0a9a3a4aea88f9) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:49.637894 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1cd724-86c7-41c8-81f7-12d71ee27a63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:49.640721 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1cd724-86c7-41c8-81f7-12d71ee27a63 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:49.655457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 48/97] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:45:49 2022] GET /identity/v3/role_assignments?user.id=7b1c7d019a3b49b79a0a9a3a4aea88f9&scope.project.id=6404c767e4bf43c8aa64c294fd5de203&role.id=7519356859fb40508c25bec8161207fd => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:50.950960 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d6f254-ba33-4fe7-bd0e-f0cb94e54535 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:50.951487 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d6f254-ba33-4fe7-bd0e-f0cb94e54535 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:50.951583 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d6f254-ba33-4fe7-bd0e-f0cb94e54535 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:50.953012 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 50/98] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:50.958666 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225fe4c-a3ed-472c-b30d-17ac8deda4d3 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:50.959128 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225fe4c-a3ed-472c-b30d-17ac8deda4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:50.959311 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225fe4c-a3ed-472c-b30d-17ac8deda4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:50.984669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-0225fe4c-a3ed-472c-b30d-17ac8deda4d3 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:50.985892 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-0225fe4c-a3ed-472c-b30d-17ac8deda4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:51.030421 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:51.066527 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 49/99] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:45:51.101692 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d6f254-ba33-4fe7-bd0e-f0cb94e54535 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:51.239301 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d6f254-ba33-4fe7-bd0e-f0cb94e54535 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:51.285733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85386d02-2220-4bfc-b1a7-0807a28f4448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:51.286406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85386d02-2220-4bfc-b1a7-0807a28f4448 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:51.286699 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85386d02-2220-4bfc-b1a7-0807a28f4448 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:51.286978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85386d02-2220-4bfc-b1a7-0807a28f4448 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:51.298699 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eafce2-954b-4e3b-a2bc-67b5c60022d7 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:51.299002 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eafce2-954b-4e3b-a2bc-67b5c60022d7 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:51.299258 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eafce2-954b-4e3b-a2bc-67b5c60022d7 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:51.300993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 50/100] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:51.301173 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85386d02-2220-4bfc-b1a7-0807a28f4448 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:45:51.306871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-85386d02-2220-4bfc-b1a7-0807a28f4448 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:45:51.306871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:45:51.306871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:45:51.306871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:45:51.306871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:45:51.306871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:45:51.306871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:45:51.306871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:45:51.306871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:45:51.306871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:45:51.306871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:45:51.306871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:45:51.306871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:45:51.306871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:45:51.306871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:45:51.306871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:45:51.307822 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:45:51.307822 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:45:51.307822 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:45:51.307822 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:45:51.307822 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:45:51.307822 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:45:51.307822 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:45:51.307822 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:45:51.307822 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:45:51.307822 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:45:51.307822 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:45:51.307822 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:45:51.307822 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:45:51.307822 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:45:51.307822 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:45:51.307822 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:45:51.308634 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:45:51.308634 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:45:51.308634 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:45:51.308634 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:45:51.308634 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:45:51.308634 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:45:51.308634 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:45:51.308634 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:45:51.308634 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:45:51.308634 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:45:51.308634 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:45:51.308634 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:45:51.308634 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 51/101] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:45:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:45:51.314816 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eafce2-954b-4e3b-a2bc-67b5c60022d7 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:51.316008 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46e99c-df14-4d35-a651-78d5d30d4e91 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:51.316311 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46e99c-df14-4d35-a651-78d5d30d4e91 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:51.316597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46e99c-df14-4d35-a651-78d5d30d4e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:51.352885 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-bd46e99c-df14-4d35-a651-78d5d30d4e91 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:51.354191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bd46e99c-df14-4d35-a651-78d5d30d4e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:51.450509 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:51.501582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eafce2-954b-4e3b-a2bc-67b5c60022d7 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:51.501993 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 52/102] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 04:45:51.535003 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46e99c-df14-4d35-a651-78d5d30d4e91 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:51.537259 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ec54b-ddc1-4b03-9e82-7c487c23575e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:51.538076 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ec54b-ddc1-4b03-9e82-7c487c23575e None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:51.538503 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ec54b-ddc1-4b03-9e82-7c487c23575e None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:51.538919 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ec54b-ddc1-4b03-9e82-7c487c23575e None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:51.539898 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ec54b-ddc1-4b03-9e82-7c487c23575e None admin] RBAC: Adding query filter params (name=member) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:51.540344 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ec54b-ddc1-4b03-9e82-7c487c23575e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:51.541902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ec54b-ddc1-4b03-9e82-7c487c23575e None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:51.551381 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 51/103] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:45:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:51.557436 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ec54b-ddc1-4b03-9e82-7c487c23575e None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:51.594898 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:51.709125 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ec54b-ddc1-4b03-9e82-7c487c23575e None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:51.711576 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46e99c-df14-4d35-a651-78d5d30d4e91 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:51.750196 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c45a15-48c1-4b82-9591-3ab92342639c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:51.751224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c45a15-48c1-4b82-9591-3ab92342639c None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:51.752373 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c45a15-48c1-4b82-9591-3ab92342639c None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:51.752878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c45a15-48c1-4b82-9591-3ab92342639c None admin] PATH_INFO: `/v3/users/7b1c7d019a3b49b79a0a9a3a4aea88f9` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:51.754425 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c45a15-48c1-4b82-9591-3ab92342639c None admin] RBAC: Authorizing `identity:get_user(user_id=7b1c7d019a3b49b79a0a9a3a4aea88f9)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:51.756151 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c45a15-48c1-4b82-9591-3ab92342639c None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:51.758603 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 52/104] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:45:51 2022] GET /identity/v3/users/7b1c7d019a3b49b79a0a9a3a4aea88f9 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:51.760602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058f1be-95bd-4d23-905a-728bda1400ae None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:51.761103 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058f1be-95bd-4d23-905a-728bda1400ae None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:51.761578 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058f1be-95bd-4d23-905a-728bda1400ae None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:51.763300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 53/105] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:51.767768 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058f1be-95bd-4d23-905a-728bda1400ae None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:51.789396 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08809d4-a0f3-4654-883d-e687f74fc0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:51.790428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08809d4-a0f3-4654-883d-e687f74fc0b1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:51.790910 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08809d4-a0f3-4654-883d-e687f74fc0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:51.791379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08809d4-a0f3-4654-883d-e687f74fc0b1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:51.813280 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08809d4-a0f3-4654-883d-e687f74fc0b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:45:51.820113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e08809d4-a0f3-4654-883d-e687f74fc0b1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 04:45:51.820113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:45:51.820113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:45:51.820113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:45:51.820113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:45:51.820113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:45:51.820113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:45:51.820113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:45:51.820113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:45:51.820113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:45:51.820113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:45:51.820113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:45:51.820113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:45:51.820113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:45:51.820113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:45:51.820113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:45:51.820966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:45:51.820966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:45:51.820966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:45:51.820966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:45:51.820966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:45:51.820966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:45:51.820966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:45:51.820966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:45:51.820966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:45:51.820966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:45:51.820966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:45:51.820966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:45:51.820966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:45:51.820966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:45:51.820966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:45:51.820966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:45:51.821732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:45:51.821732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:45:51.821732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:45:51.821732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:45:51.821732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:45:51.821732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:45:51.821732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:45:51.821732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:45:51.821732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:45:51.821732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:45:51.821732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 04:45:51.821732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:45:51.827192 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 53/106] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:45:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:45:51.829724 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40ce4b-f967-4523-b0af-2c16752d5462 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:51.832380 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40ce4b-f967-4523-b0af-2c16752d5462 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:51.833751 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40ce4b-f967-4523-b0af-2c16752d5462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:51.860370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4b40ce4b-f967-4523-b0af-2c16752d5462 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:51.861581 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4b40ce4b-f967-4523-b0af-2c16752d5462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:51.897070 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:51.920075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058f1be-95bd-4d23-905a-728bda1400ae None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:51.922815 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:51.942615 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 54/107] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:45:51.943502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b40ce4b-f967-4523-b0af-2c16752d5462 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:51.958779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a81b52-02e5-47ee-b8ac-0feb387c8465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:51.960252 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a81b52-02e5-47ee-b8ac-0feb387c8465 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:51.960783 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a81b52-02e5-47ee-b8ac-0feb387c8465 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:51.961309 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a81b52-02e5-47ee-b8ac-0feb387c8465 None admin] PATH_INFO: `/v3/projects/8be3583b9359412ba4129e556beef77c` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:51.962735 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a81b52-02e5-47ee-b8ac-0feb387c8465 None admin] RBAC: Authorizing `identity:get_project(project_id=8be3583b9359412ba4129e556beef77c)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:51.964477 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a81b52-02e5-47ee-b8ac-0feb387c8465 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:51.966534 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 54/108] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:45:51 2022] GET /identity/v3/projects/8be3583b9359412ba4129e556beef77c => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:51.968117 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310376a-2b41-44bf-b594-e818225f0294 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:51.968665 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310376a-2b41-44bf-b594-e818225f0294 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:51.969172 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310376a-2b41-44bf-b594-e818225f0294 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:51.971087 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 55/109] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:51.972977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5310376a-2b41-44bf-b594-e818225f0294 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:52.054744 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b40ce4b-f967-4523-b0af-2c16752d5462 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:52.089265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5310376a-2b41-44bf-b594-e818225f0294 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:52.094093 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573af7b7-a461-457b-b3db-9cc8f2750ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:52.094745 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573af7b7-a461-457b-b3db-9cc8f2750ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:52.095051 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573af7b7-a461-457b-b3db-9cc8f2750ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:52.095728 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573af7b7-a461-457b-b3db-9cc8f2750ca1 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:52.096610 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573af7b7-a461-457b-b3db-9cc8f2750ca1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:52.096921 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573af7b7-a461-457b-b3db-9cc8f2750ca1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:52.098390 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573af7b7-a461-457b-b3db-9cc8f2750ca1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:52.113404 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 55/110] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:45:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:52.113404 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ccb99-78fb-48f9-b594-bebb97900a99 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:52.113404 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ccb99-78fb-48f9-b594-bebb97900a99 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:52.113404 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ccb99-78fb-48f9-b594-bebb97900a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:52.130839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-523ccb99-78fb-48f9-b594-bebb97900a99 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:52.132408 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-523ccb99-78fb-48f9-b594-bebb97900a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:52.134456 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c820c3-99b3-4d63-a787-967f61ae31c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:52.135434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c820c3-99b3-4d63-a787-967f61ae31c9 None admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:52.135976 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c820c3-99b3-4d63-a787-967f61ae31c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:52.136454 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c820c3-99b3-4d63-a787-967f61ae31c9 None admin] PATH_INFO: `/v3/projects/8be3583b9359412ba4129e556beef77c/users/7b1c7d019a3b49b79a0a9a3a4aea88f9/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:52.138726 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c820c3-99b3-4d63-a787-967f61ae31c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=8be3583b9359412ba4129e556beef77c, user_id=7b1c7d019a3b49b79a0a9a3a4aea88f9, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:52.140748 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c820c3-99b3-4d63-a787-967f61ae31c9 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:52.158644 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 56/111] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jul 26 04:45:51 2022] PUT /identity/v3/projects/8be3583b9359412ba4129e556beef77c/users/7b1c7d019a3b49b79a0a9a3a4aea88f9/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:45:52.160135 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c820c3-99b3-4d63-a787-967f61ae31c9 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:52.162760 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:52.197256 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:52.225384 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 56/112] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:45:52.226065 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ccb99-78fb-48f9-b594-bebb97900a99 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:52.278371 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c820c3-99b3-4d63-a787-967f61ae31c9 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:52.340428 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675fa886-6423-464a-b8b8-ecb83a37d09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:52.341537 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675fa886-6423-464a-b8b8-ecb83a37d09a None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:52.342039 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675fa886-6423-464a-b8b8-ecb83a37d09a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:52.342540 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675fa886-6423-464a-b8b8-ecb83a37d09a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:52.345012 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675fa886-6423-464a-b8b8-ecb83a37d09a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:52.347226 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675fa886-6423-464a-b8b8-ecb83a37d09a None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:52.349755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 57/113] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:45:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:52.351060 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675fa886-6423-464a-b8b8-ecb83a37d09a None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:52.386710 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ccb99-78fb-48f9-b594-bebb97900a99 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:52.425077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fafde60-e3f1-44f4-be7c-706c7f8940a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:52.425796 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafde60-e3f1-44f4-be7c-706c7f8940a4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:52.426073 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafde60-e3f1-44f4-be7c-706c7f8940a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:52.426427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafde60-e3f1-44f4-be7c-706c7f8940a4 None admin] PATH_INFO: `/v3/users/14fe00abc013460abd04ea1b5f2ed9ee` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:52.442824 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fafde60-e3f1-44f4-be7c-706c7f8940a4 None admin] RBAC: Authorizing `identity:get_user(user_id=14fe00abc013460abd04ea1b5f2ed9ee)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:52.444547 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fafde60-e3f1-44f4-be7c-706c7f8940a4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:52.451083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 57/114] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:45:52 2022] GET /identity/v3/users/14fe00abc013460abd04ea1b5f2ed9ee => generated 289 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:52.451083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fafde60-e3f1-44f4-be7c-706c7f8940a4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:52.467036 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:52.481888 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:52.485746 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675fa886-6423-464a-b8b8-ecb83a37d09a None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:52.523408 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fbf0b-f348-4ed2-9dfb-c7fc9c39d3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:52.524492 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fbf0b-f348-4ed2-9dfb-c7fc9c39d3ca None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:52.524986 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fbf0b-f348-4ed2-9dfb-c7fc9c39d3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:52.525483 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fbf0b-f348-4ed2-9dfb-c7fc9c39d3ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:52.537722 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937fbf0b-f348-4ed2-9dfb-c7fc9c39d3ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:45:52.543698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-937fbf0b-f348-4ed2-9dfb-c7fc9c39d3ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:45:52.543698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:45:52.543698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:45:52.543698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:45:52.543698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:45:52.543698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:45:52.543698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:45:52.543698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:45:52.543698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:45:52.543698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:45:52.543698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:45:52.543698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:45:52.543698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:45:52.543698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:45:52.543698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:45:52.543698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:45:52.544616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:45:52.544616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:45:52.544616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:45:52.544616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:45:52.544616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:45:52.544616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:45:52.544616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:45:52.544616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:45:52.544616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:45:52.544616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:45:52.544616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:45:52.544616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:45:52.544616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:45:52.544616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:45:52.544616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:45:52.544616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:45:52.545409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:45:52.545409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:45:52.545409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:45:52.545409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:45:52.545409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:45:52.545409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:45:52.545409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:45:52.545409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:45:52.545409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:45:52.545409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:45:52.545409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:45:52.545409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:45:52.548379 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 58/115] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:45:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:45:52.549570 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fbf0b-f348-4ed2-9dfb-c7fc9c39d3ca None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:52.569621 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fafde60-e3f1-44f4-be7c-706c7f8940a4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:52.610970 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a979bbf-16ce-4b5e-b936-3f932899a018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:52.611534 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a979bbf-16ce-4b5e-b936-3f932899a018 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:52.611850 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a979bbf-16ce-4b5e-b936-3f932899a018 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:52.612284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a979bbf-16ce-4b5e-b936-3f932899a018 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:52.613461 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a979bbf-16ce-4b5e-b936-3f932899a018 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:52.614434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a979bbf-16ce-4b5e-b936-3f932899a018 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:52.647422 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 58/116] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jul 26 04:45:52 2022] POST /identity/v3/projects => generated 302 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:45:52.648339 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a979bbf-16ce-4b5e-b936-3f932899a018 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:52.682023 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fbf0b-f348-4ed2-9dfb-c7fc9c39d3ca None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:52.735883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fab29d-98f0-4fb8-b77e-30a1a6036f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:52.736919 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fab29d-98f0-4fb8-b77e-30a1a6036f7a None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:52.737236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fab29d-98f0-4fb8-b77e-30a1a6036f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:52.737519 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fab29d-98f0-4fb8-b77e-30a1a6036f7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:52.739644 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:52.782594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:52.798105 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a979bbf-16ce-4b5e-b936-3f932899a018 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:52.845002 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee29e62-16b6-4b74-bd60-7d25844c451d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:52.845227 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fab29d-98f0-4fb8-b77e-30a1a6036f7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:52.845668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee29e62-16b6-4b74-bd60-7d25844c451d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:52.845668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee29e62-16b6-4b74-bd60-7d25844c451d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:52.845668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee29e62-16b6-4b74-bd60-7d25844c451d None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:52.846065 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee29e62-16b6-4b74-bd60-7d25844c451d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:52.846163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fab29d-98f0-4fb8-b77e-30a1a6036f7a None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:52.846570 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee29e62-16b6-4b74-bd60-7d25844c451d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:52.849904 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee29e62-16b6-4b74-bd60-7d25844c451d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:52.859831 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 59/117] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:45:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:52.861554 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab2c09-dd8f-469a-b67f-7a1189eff7b6 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:52.862022 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab2c09-dd8f-469a-b67f-7a1189eff7b6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:52.862147 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab2c09-dd8f-469a-b67f-7a1189eff7b6 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:52.866785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 60/118] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:52.866785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b80f1-87b9-4fb9-bd9c-b780ac7bd240 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:52.866785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b80f1-87b9-4fb9-bd9c-b780ac7bd240 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:52.866785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b80f1-87b9-4fb9-bd9c-b780ac7bd240 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:52.866785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 61/119] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:52.870727 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b80f1-87b9-4fb9-bd9c-b780ac7bd240 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:52.974982 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 59/120] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:45:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:45:52.976441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a5a65-8c1a-4fc0-9318-3c5db098d90a None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:52.976814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a5a65-8c1a-4fc0-9318-3c5db098d90a None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:52.977161 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a5a65-8c1a-4fc0-9318-3c5db098d90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:52.996867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-d58a5a65-8c1a-4fc0-9318-3c5db098d90a None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:52.998183 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-d58a5a65-8c1a-4fc0-9318-3c5db098d90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:53.015330 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b80f1-87b9-4fb9-bd9c-b780ac7bd240 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:53.052938 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:53.062738 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97c2a3-b4da-4d16-b756-9c472f7c8907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:53.064399 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97c2a3-b4da-4d16-b756-9c472f7c8907 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:53.064853 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97c2a3-b4da-4d16-b756-9c472f7c8907 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:53.065528 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97c2a3-b4da-4d16-b756-9c472f7c8907 None admin] PATH_INFO: `/v3/users/1baa40dae37f4b3298a95bdc939c2f42` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:53.067355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97c2a3-b4da-4d16-b756-9c472f7c8907 None admin] RBAC: Authorizing `identity:get_user(user_id=1baa40dae37f4b3298a95bdc939c2f42)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:53.069043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97c2a3-b4da-4d16-b756-9c472f7c8907 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:53.071596 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 62/121] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:45:52 2022] GET /identity/v3/users/1baa40dae37f4b3298a95bdc939c2f42 => generated 247 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:53.077073 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825b835-6831-46fb-8764-a12faef6cdb1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:53.077398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825b835-6831-46fb-8764-a12faef6cdb1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:53.077715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825b835-6831-46fb-8764-a12faef6cdb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:53.079984 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:53.098819 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 60/122] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:45:53.100365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a5a65-8c1a-4fc0-9318-3c5db098d90a None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:53.101009 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3825b835-6831-46fb-8764-a12faef6cdb1 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:53.102378 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3825b835-6831-46fb-8764-a12faef6cdb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:53.179019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 63/123] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:45:53.179789 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825b835-6831-46fb-8764-a12faef6cdb1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:53.240294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a5a65-8c1a-4fc0-9318-3c5db098d90a None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:53.287763 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484ec94-c368-4bf4-98fb-c9b7f9e20c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:53.288482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484ec94-c368-4bf4-98fb-c9b7f9e20c1e None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:53.288769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484ec94-c368-4bf4-98fb-c9b7f9e20c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:53.289045 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484ec94-c368-4bf4-98fb-c9b7f9e20c1e None admin] PATH_INFO: `/v3/projects/8be3583b9359412ba4129e556beef77c` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:53.290259 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3484ec94-c368-4bf4-98fb-c9b7f9e20c1e None admin] RBAC: Authorizing `identity:get_project(project_id=8be3583b9359412ba4129e556beef77c)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:53.291780 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3484ec94-c368-4bf4-98fb-c9b7f9e20c1e None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:53.293802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 61/124] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:45:53 2022] GET /identity/v3/projects/8be3583b9359412ba4129e556beef77c => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:53.294592 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484ec94-c368-4bf4-98fb-c9b7f9e20c1e None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:53.295266 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825b835-6831-46fb-8764-a12faef6cdb1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:53.325428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:53.335290 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a26d6-02a6-4354-84af-18e553ad67b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:53.336326 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a26d6-02a6-4354-84af-18e553ad67b7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:53.336902 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a26d6-02a6-4354-84af-18e553ad67b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:53.337532 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a26d6-02a6-4354-84af-18e553ad67b7 None admin] PATH_INFO: `/v3/projects/8be3583b9359412ba4129e556beef77c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:53.339631 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a26d6-02a6-4354-84af-18e553ad67b7 None admin] RBAC: Authorizing `identity:get_project(project_id=8be3583b9359412ba4129e556beef77c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:53.342260 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a26d6-02a6-4354-84af-18e553ad67b7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:53.353220 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 64/125] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:45:53 2022] GET /identity/v3/projects/8be3583b9359412ba4129e556beef77c => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:53.354398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a26d6-02a6-4354-84af-18e553ad67b7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:53.359180 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:53.408950 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484ec94-c368-4bf4-98fb-c9b7f9e20c1e None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:53.446613 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d38a388-f568-40ef-a709-284aa8e42780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:53.447191 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38a388-f568-40ef-a709-284aa8e42780 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:53.447484 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38a388-f568-40ef-a709-284aa8e42780 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:53.447799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38a388-f568-40ef-a709-284aa8e42780 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:53.459829 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d38a388-f568-40ef-a709-284aa8e42780 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:45:53.461386 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a26d6-02a6-4354-84af-18e553ad67b7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:53.465144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-5d38a388-f568-40ef-a709-284aa8e42780 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:45:53.465144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:45:53.465144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:45:53.465144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:45:53.465144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:45:53.465144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:45:53.465144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:45:53.465144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:45:53.465144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:45:53.465144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:45:53.465144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:45:53.465144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:45:53.465144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:45:53.465144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:45:53.465144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:45:53.465144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:45:53.466034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:45:53.466034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:45:53.466034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:45:53.466034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:45:53.466034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:45:53.466034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:45:53.466034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:45:53.466034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:45:53.466034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:45:53.466034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:45:53.466034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:45:53.466034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:45:53.466034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:45:53.466034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:45:53.466034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:45:53.466034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:45:53.466779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:45:53.466779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:45:53.466779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:45:53.466779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:45:53.466779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:45:53.466779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:45:53.466779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:45:53.466779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:45:53.466779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:45:53.466779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:45:53.466779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:45:53.466779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:45:53.466779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 62/126] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:45:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:45:53.467406 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d38a388-f568-40ef-a709-284aa8e42780 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:53.501047 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4f689-1916-42a2-816c-929869b9fe15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:53.502150 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4f689-1916-42a2-816c-929869b9fe15 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:53.502629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4f689-1916-42a2-816c-929869b9fe15 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:53.503149 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4f689-1916-42a2-816c-929869b9fe15 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:53.514707 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4f689-1916-42a2-816c-929869b9fe15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:45:53.520279 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-9ec4f689-1916-42a2-816c-929869b9fe15 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 04:45:53.520279 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:45:53.520279 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:45:53.520279 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:45:53.520279 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:45:53.520279 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:45:53.520279 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:45:53.520279 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:45:53.520279 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:45:53.520279 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:45:53.520279 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:45:53.520279 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:45:53.520279 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:45:53.520279 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:45:53.520279 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:45:53.520279 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:45:53.521155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:45:53.521155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:45:53.521155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:45:53.521155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:45:53.521155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:45:53.521155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:45:53.521155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:45:53.521155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:45:53.521155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:45:53.521155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:45:53.521155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:45:53.521155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:45:53.521155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:45:53.521155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:45:53.521155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:45:53.521155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:45:53.521914 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:45:53.521914 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:45:53.521914 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:45:53.521914 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:45:53.521914 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:45:53.521914 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:45:53.521914 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:45:53.521914 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:45:53.521914 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:45:53.521914 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:45:53.521914 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 04:45:53.521914 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:45:53.524399 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 65/127] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jul 26 04:45:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:45:53.525623 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4f689-1916-42a2-816c-929869b9fe15 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:53.583610 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d38a388-f568-40ef-a709-284aa8e42780 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:53.587584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:53.605197 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:53.625307 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386ad81-6a39-45f9-b84e-f1ef5d708b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:53.625945 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386ad81-6a39-45f9-b84e-f1ef5d708b06 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:53.626325 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386ad81-6a39-45f9-b84e-f1ef5d708b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:53.626604 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386ad81-6a39-45f9-b84e-f1ef5d708b06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:53.627708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3386ad81-6a39-45f9-b84e-f1ef5d708b06 None admin] RBAC: Adding query filter params (role.id=6c31395d654e4add9c982f62b79c2873, scope.project.id=8be3583b9359412ba4129e556beef77c, user.id=14fe00abc013460abd04ea1b5f2ed9ee) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:53.627903 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3386ad81-6a39-45f9-b84e-f1ef5d708b06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:53.629348 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3386ad81-6a39-45f9-b84e-f1ef5d708b06 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:53.635656 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 63/128] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:45:53 2022] GET /identity/v3/role_assignments?user.id=14fe00abc013460abd04ea1b5f2ed9ee&scope.project.id=8be3583b9359412ba4129e556beef77c&role.id=6c31395d654e4add9c982f62b79c2873 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:53.636473 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386ad81-6a39-45f9-b84e-f1ef5d708b06 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:53.637523 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4f689-1916-42a2-816c-929869b9fe15 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:53.677159 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017013c0-6074-40b5-a9e8-ea8b2d9d42fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:53.678265 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017013c0-6074-40b5-a9e8-ea8b2d9d42fb None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:53.678768 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017013c0-6074-40b5-a9e8-ea8b2d9d42fb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:53.679238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017013c0-6074-40b5-a9e8-ea8b2d9d42fb None admin] PATH_INFO: `/v3/projects/8be3583b9359412ba4129e556beef77c/users/1baa40dae37f4b3298a95bdc939c2f42/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:53.681729 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017013c0-6074-40b5-a9e8-ea8b2d9d42fb None admin] RBAC: Authorizing `identity:create_grant(project_id=8be3583b9359412ba4129e556beef77c, user_id=1baa40dae37f4b3298a95bdc939c2f42, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:53.683579 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017013c0-6074-40b5-a9e8-ea8b2d9d42fb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:53.707735 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 66/129] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jul 26 04:45:53 2022] PUT /identity/v3/projects/8be3583b9359412ba4129e556beef77c/users/1baa40dae37f4b3298a95bdc939c2f42/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:45:53.708993 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017013c0-6074-40b5-a9e8-ea8b2d9d42fb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:53.754085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386ad81-6a39-45f9-b84e-f1ef5d708b06 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:53.803131 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead2756-8dd5-45a9-bede-404b75ae114c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:53.806027 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead2756-8dd5-45a9-bede-404b75ae114c None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:53.806352 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead2756-8dd5-45a9-bede-404b75ae114c None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:53.806636 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead2756-8dd5-45a9-bede-404b75ae114c None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:53.807421 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead2756-8dd5-45a9-bede-404b75ae114c None admin] RBAC: Adding query filter params (name=member) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:53.807756 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead2756-8dd5-45a9-bede-404b75ae114c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:53.809196 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead2756-8dd5-45a9-bede-404b75ae114c None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:53.820475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 64/130] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:45:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:53.827885 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead2756-8dd5-45a9-bede-404b75ae114c None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:53.881279 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017013c0-6074-40b5-a9e8-ea8b2d9d42fb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:53.901134 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:53.915913 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:53.923593 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43d906-d2b9-4227-b0f2-f144f2155908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:53.924307 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43d906-d2b9-4227-b0f2-f144f2155908 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:53.924598 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43d906-d2b9-4227-b0f2-f144f2155908 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:53.924882 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43d906-d2b9-4227-b0f2-f144f2155908 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:53.925651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43d906-d2b9-4227-b0f2-f144f2155908 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:53.925965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43d906-d2b9-4227-b0f2-f144f2155908 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:53.927464 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43d906-d2b9-4227-b0f2-f144f2155908 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:53.937445 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 67/131] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:45:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:53.943792 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43d906-d2b9-4227-b0f2-f144f2155908 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:54.036167 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead2756-8dd5-45a9-bede-404b75ae114c None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:54.114144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ebcfc-e1b8-4e34-b37f-ede38c978951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:54.114832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ebcfc-e1b8-4e34-b37f-ede38c978951 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:54.117082 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ebcfc-e1b8-4e34-b37f-ede38c978951 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:54.117410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ebcfc-e1b8-4e34-b37f-ede38c978951 None admin] PATH_INFO: `/v3/users/7b1c7d019a3b49b79a0a9a3a4aea88f9` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:54.121188 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43d906-d2b9-4227-b0f2-f144f2155908 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:54.123696 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0ebcfc-e1b8-4e34-b37f-ede38c978951 None admin] RBAC: Authorizing `identity:get_user(user_id=7b1c7d019a3b49b79a0a9a3a4aea88f9)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:54.125268 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0ebcfc-e1b8-4e34-b37f-ede38c978951 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:54.127273 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 65/132] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:45:53 2022] GET /identity/v3/users/7b1c7d019a3b49b79a0a9a3a4aea88f9 => generated 275 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:45:54.132635 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ebcfc-e1b8-4e34-b37f-ede38c978951 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:54.213255 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb717dc-299b-40ed-81c4-116f1c581c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:54.213935 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb717dc-299b-40ed-81c4-116f1c581c85 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:54.214220 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb717dc-299b-40ed-81c4-116f1c581c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:54.214494 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb717dc-299b-40ed-81c4-116f1c581c85 None admin] PATH_INFO: `/v3/users/7b1c7d019a3b49b79a0a9a3a4aea88f9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:54.220724 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb717dc-299b-40ed-81c4-116f1c581c85 None admin] RBAC: Authorizing `identity:get_user(user_id=7b1c7d019a3b49b79a0a9a3a4aea88f9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:54.222316 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb717dc-299b-40ed-81c4-116f1c581c85 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:54.224274 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 68/133] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:45:53 2022] GET /identity/v3/users/7b1c7d019a3b49b79a0a9a3a4aea88f9 => generated 275 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:45:54.229591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb717dc-299b-40ed-81c4-116f1c581c85 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:54.356156 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:54.370017 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ebcfc-e1b8-4e34-b37f-ede38c978951 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:54.397397 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:54.468449 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f65642-e990-45b8-ab71-9d0ae45c6ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:54.469139 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f65642-e990-45b8-ab71-9d0ae45c6ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:54.469431 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f65642-e990-45b8-ab71-9d0ae45c6ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:54.469715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f65642-e990-45b8-ab71-9d0ae45c6ed9 None admin] PATH_INFO: `/v3/projects/6404c767e4bf43c8aa64c294fd5de203` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:54.471375 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f65642-e990-45b8-ab71-9d0ae45c6ed9 None admin] RBAC: Authorizing `identity:get_project(project_id=6404c767e4bf43c8aa64c294fd5de203)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:54.472903 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f65642-e990-45b8-ab71-9d0ae45c6ed9 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:54.474699 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 66/134] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:45:54 2022] GET /identity/v3/projects/6404c767e4bf43c8aa64c294fd5de203 => generated 312 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:54.485061 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f65642-e990-45b8-ab71-9d0ae45c6ed9 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:54.503244 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb717dc-299b-40ed-81c4-116f1c581c85 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:54.580362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c0540-4440-463c-a4b4-e6b0008dc6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:54.581056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c0540-4440-463c-a4b4-e6b0008dc6a8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:54.581342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c0540-4440-463c-a4b4-e6b0008dc6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:54.581617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c0540-4440-463c-a4b4-e6b0008dc6a8 None admin] PATH_INFO: `/v3/projects/8be3583b9359412ba4129e556beef77c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:54.582913 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c0540-4440-463c-a4b4-e6b0008dc6a8 None admin] RBAC: Authorizing `identity:get_project(project_id=8be3583b9359412ba4129e556beef77c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:54.584432 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c0540-4440-463c-a4b4-e6b0008dc6a8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:54.586879 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 69/135] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:45:54 2022] GET /identity/v3/projects/8be3583b9359412ba4129e556beef77c => generated 298 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:45:54.592513 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c0540-4440-463c-a4b4-e6b0008dc6a8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:54.713001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f65642-e990-45b8-ab71-9d0ae45c6ed9 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:54.797715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde6f502-a5d7-4f09-9c6a-db5e3bb74eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:54.798423 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6f502-a5d7-4f09-9c6a-db5e3bb74eef None admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:54.798710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6f502-a5d7-4f09-9c6a-db5e3bb74eef None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:54.798993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6f502-a5d7-4f09-9c6a-db5e3bb74eef None admin] PATH_INFO: `/v3/projects/6404c767e4bf43c8aa64c294fd5de203/users/7b1c7d019a3b49b79a0a9a3a4aea88f9/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:54.801980 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c0540-4440-463c-a4b4-e6b0008dc6a8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:54.805726 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde6f502-a5d7-4f09-9c6a-db5e3bb74eef None admin] RBAC: Authorizing `identity:create_grant(project_id=6404c767e4bf43c8aa64c294fd5de203, user_id=7b1c7d019a3b49b79a0a9a3a4aea88f9, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:54.807237 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde6f502-a5d7-4f09-9c6a-db5e3bb74eef None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:54.823100 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:54.832538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 67/136] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jul 26 04:45:54 2022] PUT /identity/v3/projects/6404c767e4bf43c8aa64c294fd5de203/users/7b1c7d019a3b49b79a0a9a3a4aea88f9/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:45:54.834016 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac627d-d8cf-45ce-afbf-df5b9220604e None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:54.834314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac627d-d8cf-45ce-afbf-df5b9220604e None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:54.834595 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac627d-d8cf-45ce-afbf-df5b9220604e None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:54.836037 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 68/137] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:54.848289 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d07f15-9c8d-4607-9a74-75f53f0f7e5c None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:54.848623 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d07f15-9c8d-4607-9a74-75f53f0f7e5c None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:54.848865 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d07f15-9c8d-4607-9a74-75f53f0f7e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:54.883778 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-12d07f15-9c8d-4607-9a74-75f53f0f7e5c None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:54.885019 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-12d07f15-9c8d-4607-9a74-75f53f0f7e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:54.888192 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95ff1d-725c-4e1a-abda-38f6e8700aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:54.888934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95ff1d-725c-4e1a-abda-38f6e8700aa5 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:54.889221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95ff1d-725c-4e1a-abda-38f6e8700aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:54.889500 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95ff1d-725c-4e1a-abda-38f6e8700aa5 None admin] PATH_INFO: `/v3/projects/8be3583b9359412ba4129e556beef77c/users/7b1c7d019a3b49b79a0a9a3a4aea88f9/roles/9223a9bb40484faca3a7bcfeb41a2215` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:54.892163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc95ff1d-725c-4e1a-abda-38f6e8700aa5 None admin] RBAC: Authorizing `identity:create_grant(project_id=8be3583b9359412ba4129e556beef77c, user_id=7b1c7d019a3b49b79a0a9a3a4aea88f9, role_id=9223a9bb40484faca3a7bcfeb41a2215)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:54.893679 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc95ff1d-725c-4e1a-abda-38f6e8700aa5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:54.917583 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 70/138] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jul 26 04:45:54 2022] PUT /identity/v3/projects/8be3583b9359412ba4129e556beef77c/users/7b1c7d019a3b49b79a0a9a3a4aea88f9/roles/9223a9bb40484faca3a7bcfeb41a2215 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:45:54.972274 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:54.980332 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 69/139] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 04:45:55.004174 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95ff1d-725c-4e1a-abda-38f6e8700aa5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:55.146773 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95ff1d-725c-4e1a-abda-38f6e8700aa5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:55.190423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d77ca-7e72-421d-a906-d952a02a564f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:55.191114 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d77ca-7e72-421d-a906-d952a02a564f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:55.191400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d77ca-7e72-421d-a906-d952a02a564f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:55.191703 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d77ca-7e72-421d-a906-d952a02a564f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:55.209339 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d77ca-7e72-421d-a906-d952a02a564f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:45:55.214934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-697d77ca-7e72-421d-a906-d952a02a564f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:45:55.214934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:45:55.214934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:45:55.214934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:45:55.214934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:45:55.214934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:45:55.214934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:45:55.214934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:45:55.214934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:45:55.214934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:45:55.214934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:45:55.214934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:45:55.214934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:45:55.214934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:45:55.214934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:45:55.214934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:45:55.215862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:45:55.215862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:45:55.215862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:45:55.215862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:45:55.215862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:45:55.215862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:45:55.215862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:45:55.215862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:45:55.215862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:45:55.215862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:45:55.215862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:45:55.215862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:45:55.215862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:45:55.215862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:45:55.215862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:45:55.215862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:45:55.216584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:45:55.216584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:45:55.216584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:45:55.216584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:45:55.216584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:45:55.216584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:45:55.216584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:45:55.216584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:45:55.216584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:45:55.216584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:45:55.216584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:45:55.216584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:45:55.216584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 71/140] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:45:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:45:55.222895 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d07f15-9c8d-4607-9a74-75f53f0f7e5c None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:55.342374 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959a37d-42ef-43e1-9ed7-14327feb8c2c None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:55.342729 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959a37d-42ef-43e1-9ed7-14327feb8c2c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:55.342955 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959a37d-42ef-43e1-9ed7-14327feb8c2c None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:55.345980 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 72/141] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:55.352740 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e4f64-b0d9-4a11-9b8f-d08d8ec62b97 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:55.353032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e4f64-b0d9-4a11-9b8f-d08d8ec62b97 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:55.353661 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e4f64-b0d9-4a11-9b8f-d08d8ec62b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:55.361186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d07f15-9c8d-4607-9a74-75f53f0f7e5c None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:55.376129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f31e4f64-b0d9-4a11-9b8f-d08d8ec62b97 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:55.377354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f31e4f64-b0d9-4a11-9b8f-d08d8ec62b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:55.444505 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27856e-bace-4f99-a7c5-ff59b85b7200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:55.445292 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27856e-bace-4f99-a7c5-ff59b85b7200 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:55.445585 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27856e-bace-4f99-a7c5-ff59b85b7200 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:55.445866 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27856e-bace-4f99-a7c5-ff59b85b7200 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:55.446640 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27856e-bace-4f99-a7c5-ff59b85b7200 None admin] RBAC: Adding query filter params (name=member) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:55.446954 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27856e-bace-4f99-a7c5-ff59b85b7200 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:55.448841 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:55.449614 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27856e-bace-4f99-a7c5-ff59b85b7200 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:55.473984 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 70/142] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:45:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:55.479946 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27856e-bace-4f99-a7c5-ff59b85b7200 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:55.510061 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 73/143] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:45:55.542805 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:55.558468 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e4f64-b0d9-4a11-9b8f-d08d8ec62b97 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:55.638494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27856e-bace-4f99-a7c5-ff59b85b7200 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:55.678712 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94851b0b-7587-4fc8-aa81-f3108285cc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:55.679487 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94851b0b-7587-4fc8-aa81-f3108285cc9a None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:55.680018 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94851b0b-7587-4fc8-aa81-f3108285cc9a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:55.680310 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94851b0b-7587-4fc8-aa81-f3108285cc9a None admin] PATH_INFO: `/v3/users/7b1c7d019a3b49b79a0a9a3a4aea88f9` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:55.681838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94851b0b-7587-4fc8-aa81-f3108285cc9a None admin] RBAC: Authorizing `identity:get_user(user_id=7b1c7d019a3b49b79a0a9a3a4aea88f9)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:55.683936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94851b0b-7587-4fc8-aa81-f3108285cc9a None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:55.685723 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 71/144] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:45:55 2022] GET /identity/v3/users/7b1c7d019a3b49b79a0a9a3a4aea88f9 => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:55.691629 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94851b0b-7587-4fc8-aa81-f3108285cc9a None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:55.769821 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e4f64-b0d9-4a11-9b8f-d08d8ec62b97 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:55.844584 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:55.855902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94851b0b-7587-4fc8-aa81-f3108285cc9a None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:55.888413 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed169b6-b231-4493-8436-124a8fa2dee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:55.889105 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed169b6-b231-4493-8436-124a8fa2dee1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:55.889393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed169b6-b231-4493-8436-124a8fa2dee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:55.889674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed169b6-b231-4493-8436-124a8fa2dee1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:55.891022 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed169b6-b231-4493-8436-124a8fa2dee1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:55.892540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed169b6-b231-4493-8436-124a8fa2dee1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:55.894506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 74/145] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:45:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:45:55.903584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed169b6-b231-4493-8436-124a8fa2dee1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:55.909168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4954c9d-6a54-4199-8ef7-b2cefc569d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:55.909821 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4954c9d-6a54-4199-8ef7-b2cefc569d11 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:55.910106 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4954c9d-6a54-4199-8ef7-b2cefc569d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:55.910384 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4954c9d-6a54-4199-8ef7-b2cefc569d11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:55.989684 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:56.098998 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4954c9d-6a54-4199-8ef7-b2cefc569d11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:56.100929 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4954c9d-6a54-4199-8ef7-b2cefc569d11 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:56.155816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed169b6-b231-4493-8436-124a8fa2dee1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:56.248349 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4a547-9cba-49ea-b5ec-556683610def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:56.249019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4a547-9cba-49ea-b5ec-556683610def None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:56.249466 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4a547-9cba-49ea-b5ec-556683610def None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:56.249747 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4a547-9cba-49ea-b5ec-556683610def None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:56.250469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4a547-9cba-49ea-b5ec-556683610def None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:56.251982 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4a547-9cba-49ea-b5ec-556683610def None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:56.275460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:56.283032 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 72/146] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:45:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:45:56.297656 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4954c9d-6a54-4199-8ef7-b2cefc569d11 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:56.300944 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 75/147] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jul 26 04:45:55 2022] POST /identity/v3/users => generated 283 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:45:56.444755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4954c9d-6a54-4199-8ef7-b2cefc569d11 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:56.496192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2775262-a3be-4bfa-a452-2b6b85826fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:56.496867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2775262-a3be-4bfa-a452-2b6b85826fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:56.497149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2775262-a3be-4bfa-a452-2b6b85826fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:56.497426 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2775262-a3be-4bfa-a452-2b6b85826fa3 None admin] PATH_INFO: `/v3/projects/8be3583b9359412ba4129e556beef77c` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:56.498600 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2775262-a3be-4bfa-a452-2b6b85826fa3 None admin] RBAC: Authorizing `identity:get_project(project_id=8be3583b9359412ba4129e556beef77c)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:56.500252 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2775262-a3be-4bfa-a452-2b6b85826fa3 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:56.502130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 73/148] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:45:56 2022] GET /identity/v3/projects/8be3583b9359412ba4129e556beef77c => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:45:56.508306 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4a547-9cba-49ea-b5ec-556683610def None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:56.648637 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:56.663353 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4a547-9cba-49ea-b5ec-556683610def None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:56.701412 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e03c5-6373-4f28-82fb-685635c011bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:56.702050 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e03c5-6373-4f28-82fb-685635c011bf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:56.702331 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e03c5-6373-4f28-82fb-685635c011bf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:56.702666 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e03c5-6373-4f28-82fb-685635c011bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:56.703473 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08e03c5-6373-4f28-82fb-685635c011bf None admin] RBAC: Adding query filter params (role.id=7519356859fb40508c25bec8161207fd, scope.project.id=8be3583b9359412ba4129e556beef77c, user.id=7b1c7d019a3b49b79a0a9a3a4aea88f9) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:56.703814 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08e03c5-6373-4f28-82fb-685635c011bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:56.705352 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08e03c5-6373-4f28-82fb-685635c011bf None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:56.712152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 76/149] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:45:56 2022] GET /identity/v3/role_assignments?user.id=7b1c7d019a3b49b79a0a9a3a4aea88f9&scope.project.id=8be3583b9359412ba4129e556beef77c&role.id=7519356859fb40508c25bec8161207fd => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:56.759637 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaeda7b-f83c-43f7-aa9b-774f80183c8d None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:56.759944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaeda7b-f83c-43f7-aa9b-774f80183c8d None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:56.760205 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaeda7b-f83c-43f7-aa9b-774f80183c8d None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:56.761937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 74/150] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:56.767439 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7409ba-65f4-40f5-b2ef-ad7bc2beea50 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:56.767806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7409ba-65f4-40f5-b2ef-ad7bc2beea50 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:56.768032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7409ba-65f4-40f5-b2ef-ad7bc2beea50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:56.793501 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9a7409ba-65f4-40f5-b2ef-ad7bc2beea50 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:56.794163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727a346-f847-4910-a0f7-f35f6b60c812 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:56.794447 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727a346-f847-4910-a0f7-f35f6b60c812 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:56.794732 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727a346-f847-4910-a0f7-f35f6b60c812 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:56.794896 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9a7409ba-65f4-40f5-b2ef-ad7bc2beea50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:56.796336 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 75/151] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:45:56.801525 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984e69d-ddda-47ea-a02a-c6fc0d503b83 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:56.801806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984e69d-ddda-47ea-a02a-c6fc0d503b83 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:56.802073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984e69d-ddda-47ea-a02a-c6fc0d503b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:56.840507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-5984e69d-ddda-47ea-a02a-c6fc0d503b83 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:56.841719 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-5984e69d-ddda-47ea-a02a-c6fc0d503b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:56.880501 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 77/152] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:45:56.902409 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7409ba-65f4-40f5-b2ef-ad7bc2beea50 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:56.946587 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:56.952044 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 76/153] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 04:45:56.978558 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984e69d-ddda-47ea-a02a-c6fc0d503b83 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:56.993784 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:57.016467 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7409ba-65f4-40f5-b2ef-ad7bc2beea50 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:57.054282 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85c16f-bdab-4cc7-94bf-10c2fc220304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:57.054952 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85c16f-bdab-4cc7-94bf-10c2fc220304 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:57.055236 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85c16f-bdab-4cc7-94bf-10c2fc220304 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:57.055513 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85c16f-bdab-4cc7-94bf-10c2fc220304 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:57.067155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85c16f-bdab-4cc7-94bf-10c2fc220304 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:45:57.072164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-3d85c16f-bdab-4cc7-94bf-10c2fc220304 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 04:45:57.072164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:45:57.072164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:45:57.072164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:45:57.072164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:45:57.072164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:45:57.072164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:45:57.072164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:45:57.072164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:45:57.072164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:45:57.072164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:45:57.072164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:45:57.072164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:45:57.072164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:45:57.072164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:45:57.072164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:45:57.073032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:45:57.073032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:45:57.073032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:45:57.073032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:45:57.073032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:45:57.073032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:45:57.073032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:45:57.073032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:45:57.073032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:45:57.073032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:45:57.073032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:45:57.073032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:45:57.073032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:45:57.073032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:45:57.073032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:45:57.073032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:45:57.073785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:45:57.073785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:45:57.073785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:45:57.073785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:45:57.073785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:45:57.073785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:45:57.073785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:45:57.073785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:45:57.073785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:45:57.073785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:45:57.073785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 04:45:57.073785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:45:57.073785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 78/154] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:45:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:45:57.084653 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85c16f-bdab-4cc7-94bf-10c2fc220304 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:57.132464 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984e69d-ddda-47ea-a02a-c6fc0d503b83 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:57.192975 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9794c6b-8ad1-41fa-ab33-fa875944e546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:57.193560 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9794c6b-8ad1-41fa-ab33-fa875944e546 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:57.193845 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9794c6b-8ad1-41fa-ab33-fa875944e546 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:57.194118 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9794c6b-8ad1-41fa-ab33-fa875944e546 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:57.209903 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9794c6b-8ad1-41fa-ab33-fa875944e546 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:45:57.214832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-b9794c6b-8ad1-41fa-ab33-fa875944e546 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:45:57.214832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:45:57.214832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:45:57.214832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:45:57.214832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:45:57.214832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:45:57.214832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:45:57.214832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:45:57.214832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:45:57.214832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:45:57.214832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:45:57.214832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:45:57.214832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:45:57.214832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:45:57.214832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:45:57.214832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:45:57.215690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:45:57.215690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:45:57.215690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:45:57.215690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:45:57.215690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:45:57.215690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:45:57.215690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:45:57.215690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:45:57.215690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:45:57.215690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:45:57.215690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:45:57.215690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:45:57.215690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:45:57.215690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:45:57.215690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:45:57.215690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:45:57.216992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:45:57.216992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:45:57.216992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:45:57.216992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:45:57.216992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:45:57.216992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:45:57.216992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:45:57.216992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:45:57.216992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:45:57.216992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:45:57.216992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:45:57.216992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:45:57.216992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 77/155] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:45:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:45:57.223133 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9794c6b-8ad1-41fa-ab33-fa875944e546 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:57.235945 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85c16f-bdab-4cc7-94bf-10c2fc220304 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:57.284680 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552b094-84ce-4616-b70f-ee1739288067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:57.285338 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552b094-84ce-4616-b70f-ee1739288067 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:57.285619 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552b094-84ce-4616-b70f-ee1739288067 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:57.285894 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552b094-84ce-4616-b70f-ee1739288067 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:57.286937 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552b094-84ce-4616-b70f-ee1739288067 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:57.287301 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552b094-84ce-4616-b70f-ee1739288067 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:57.289450 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552b094-84ce-4616-b70f-ee1739288067 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:57.301491 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 79/156] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:45:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:57.307767 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:57.308501 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552b094-84ce-4616-b70f-ee1739288067 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:57.314442 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:57.362331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9794c6b-8ad1-41fa-ab33-fa875944e546 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:57.408438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d2806-e78a-4fa7-b63a-c16dde2725cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:57.409402 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d2806-e78a-4fa7-b63a-c16dde2725cc None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:57.409825 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d2806-e78a-4fa7-b63a-c16dde2725cc None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:57.410272 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d2806-e78a-4fa7-b63a-c16dde2725cc None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:57.411217 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d2806-e78a-4fa7-b63a-c16dde2725cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:57.411743 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d2806-e78a-4fa7-b63a-c16dde2725cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:57.413406 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d2806-e78a-4fa7-b63a-c16dde2725cc None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:57.430800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 78/157] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:45:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:57.443520 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d2806-e78a-4fa7-b63a-c16dde2725cc None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:57.467784 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552b094-84ce-4616-b70f-ee1739288067 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:57.538391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84e31c-4785-49d4-8a2a-dc9222414576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:57.539447 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84e31c-4785-49d4-8a2a-dc9222414576 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:57.539898 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84e31c-4785-49d4-8a2a-dc9222414576 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:57.540386 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84e31c-4785-49d4-8a2a-dc9222414576 None admin] PATH_INFO: `/v3/users/14fe00abc013460abd04ea1b5f2ed9ee` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:57.542079 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d84e31c-4785-49d4-8a2a-dc9222414576 None admin] RBAC: Authorizing `identity:get_user(user_id=14fe00abc013460abd04ea1b5f2ed9ee)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:57.543837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d84e31c-4785-49d4-8a2a-dc9222414576 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:57.545768 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 80/158] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:45:57 2022] GET /identity/v3/users/14fe00abc013460abd04ea1b5f2ed9ee => generated 289 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:57.551842 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84e31c-4785-49d4-8a2a-dc9222414576 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:57.600802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d2806-e78a-4fa7-b63a-c16dde2725cc None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:57.622714 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:57.639187 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fccdb3-65aa-4512-a964-46959525d4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:57.639904 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fccdb3-65aa-4512-a964-46959525d4bc None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:57.640188 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fccdb3-65aa-4512-a964-46959525d4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:57.640461 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fccdb3-65aa-4512-a964-46959525d4bc None admin] PATH_INFO: `/v3/users/1baa40dae37f4b3298a95bdc939c2f42` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:57.641859 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fccdb3-65aa-4512-a964-46959525d4bc None admin] RBAC: Authorizing `identity:get_user(user_id=1baa40dae37f4b3298a95bdc939c2f42)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:57.643341 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fccdb3-65aa-4512-a964-46959525d4bc None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:57.645399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 79/159] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:45:57 2022] GET /identity/v3/users/1baa40dae37f4b3298a95bdc939c2f42 => generated 247 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:45:57.651089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fccdb3-65aa-4512-a964-46959525d4bc None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:57.670346 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:57.711287 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84e31c-4785-49d4-8a2a-dc9222414576 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:57.763582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c57f5-772a-47cd-80bd-14265045fa6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:57.764576 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c57f5-772a-47cd-80bd-14265045fa6b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:57.765035 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c57f5-772a-47cd-80bd-14265045fa6b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:57.765426 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c57f5-772a-47cd-80bd-14265045fa6b None admin] PATH_INFO: `/v3/projects/8be3583b9359412ba4129e556beef77c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:57.766741 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c57f5-772a-47cd-80bd-14265045fa6b None admin] RBAC: Authorizing `identity:get_project(project_id=8be3583b9359412ba4129e556beef77c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:57.767238 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fccdb3-65aa-4512-a964-46959525d4bc None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:57.768462 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c57f5-772a-47cd-80bd-14265045fa6b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:57.771062 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 81/160] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:45:57 2022] GET /identity/v3/projects/8be3583b9359412ba4129e556beef77c => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:57.780698 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c57f5-772a-47cd-80bd-14265045fa6b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:57.837027 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717c9f5-d678-4536-8496-77dd910c1325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:57.838513 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717c9f5-d678-4536-8496-77dd910c1325 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:57.843847 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717c9f5-d678-4536-8496-77dd910c1325 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:57.844844 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717c9f5-d678-4536-8496-77dd910c1325 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:57.957138 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:57.971143 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f717c9f5-d678-4536-8496-77dd910c1325 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:57.972904 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f717c9f5-d678-4536-8496-77dd910c1325 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:58.028205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c57f5-772a-47cd-80bd-14265045fa6b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:58.080650 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 80/161] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:45:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 04:45:58.082922 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f9b14-765e-4e70-80a3-31812d0c78b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:58.083851 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f9b14-765e-4e70-80a3-31812d0c78b5 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:58.084276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f9b14-765e-4e70-80a3-31812d0c78b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:58.084688 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f9b14-765e-4e70-80a3-31812d0c78b5 None admin] PATH_INFO: `/v3/projects/8be3583b9359412ba4129e556beef77c/users/14fe00abc013460abd04ea1b5f2ed9ee/roles/6c31395d654e4add9c982f62b79c2873` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:58.087386 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717c9f5-d678-4536-8496-77dd910c1325 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:58.097611 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f9b14-765e-4e70-80a3-31812d0c78b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=8be3583b9359412ba4129e556beef77c, user_id=14fe00abc013460abd04ea1b5f2ed9ee, role_id=6c31395d654e4add9c982f62b79c2873)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:58.099374 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f9b14-765e-4e70-80a3-31812d0c78b5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:58.117909 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:58.121093 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 82/162] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jul 26 04:45:57 2022] PUT /identity/v3/projects/8be3583b9359412ba4129e556beef77c/users/14fe00abc013460abd04ea1b5f2ed9ee/roles/6c31395d654e4add9c982f62b79c2873 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 04:45:58.199250 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:58.212321 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717c9f5-d678-4536-8496-77dd910c1325 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:58.250806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2740fb38-937c-489c-a414-711c9003d299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:58.251394 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740fb38-937c-489c-a414-711c9003d299 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:58.251721 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740fb38-937c-489c-a414-711c9003d299 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:58.252004 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740fb38-937c-489c-a414-711c9003d299 None admin] PATH_INFO: `/v3/projects/8be3583b9359412ba4129e556beef77c` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:58.253230 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2740fb38-937c-489c-a414-711c9003d299 None admin] RBAC: Authorizing `identity:get_project(project_id=8be3583b9359412ba4129e556beef77c)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:58.254708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2740fb38-937c-489c-a414-711c9003d299 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:58.256856 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 81/163] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:45:58 2022] GET /identity/v3/projects/8be3583b9359412ba4129e556beef77c => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:58.262780 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f9b14-765e-4e70-80a3-31812d0c78b5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:58.298912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd63831-7079-4b80-859d-e121a0938259 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:58.299269 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd63831-7079-4b80-859d-e121a0938259 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:58.299459 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd63831-7079-4b80-859d-e121a0938259 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:58.301028 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 82/164] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:45:58.306635 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32219b-d079-4464-aef3-7ef0ddf3a5cd None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:58.306948 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32219b-d079-4464-aef3-7ef0ddf3a5cd None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:58.307198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32219b-d079-4464-aef3-7ef0ddf3a5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:58.343468 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-6b32219b-d079-4464-aef3-7ef0ddf3a5cd None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:58.344751 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-6b32219b-d079-4464-aef3-7ef0ddf3a5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:58.486893 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f9b14-765e-4e70-80a3-31812d0c78b5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:58.499501 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 83/165] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 04:45:58.500731 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629dd61-98fc-4a54-a861-a4bf0862e5ca None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:58.501069 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629dd61-98fc-4a54-a861-a4bf0862e5ca None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:58.501297 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629dd61-98fc-4a54-a861-a4bf0862e5ca None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:58.503354 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 84/166] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:45:58.511809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8fb2d-278a-404c-9714-7d2969989de5 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:58.512124 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8fb2d-278a-404c-9714-7d2969989de5 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:58.512362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8fb2d-278a-404c-9714-7d2969989de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:58.545658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-cfa8fb2d-278a-404c-9714-7d2969989de5 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:45:58.546907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-cfa8fb2d-278a-404c-9714-7d2969989de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:45:58.562028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f9190e-1255-44d2-8732-e3fd573d8b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:58.562841 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9190e-1255-44d2-8732-e3fd573d8b49 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:58.563027 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9190e-1255-44d2-8732-e3fd573d8b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:58.563397 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9190e-1255-44d2-8732-e3fd573d8b49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:58.564324 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f9190e-1255-44d2-8732-e3fd573d8b49 None admin] RBAC: Adding query filter params (role.id=72bf8bd23d2c4b76ad32b7b35d6db412, scope.project.id=8be3583b9359412ba4129e556beef77c, user.id=1baa40dae37f4b3298a95bdc939c2f42) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:58.564643 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f9190e-1255-44d2-8732-e3fd573d8b49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:58.566209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f9190e-1255-44d2-8732-e3fd573d8b49 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:58.577935 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 83/167] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:45:58 2022] GET /identity/v3/role_assignments?user.id=1baa40dae37f4b3298a95bdc939c2f42&scope.project.id=8be3583b9359412ba4129e556beef77c&role.id=72bf8bd23d2c4b76ad32b7b35d6db412 => generated 608 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:58.578834 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f9190e-1255-44d2-8732-e3fd573d8b49 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:58.652326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 85/168] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:45:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:45:58.692360 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:58.694100 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8fb2d-278a-404c-9714-7d2969989de5 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:58.734160 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:58.752723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f9190e-1255-44d2-8732-e3fd573d8b49 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:58.820513 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29018b00-c253-4345-9b77-aab17189350b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:58.821194 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29018b00-c253-4345-9b77-aab17189350b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:58.821478 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29018b00-c253-4345-9b77-aab17189350b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:58.821818 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29018b00-c253-4345-9b77-aab17189350b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:58.849378 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29018b00-c253-4345-9b77-aab17189350b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:45:58.855979 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-29018b00-c253-4345-9b77-aab17189350b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 04:45:58.855979 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:45:58.855979 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:45:58.855979 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:45:58.855979 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:45:58.855979 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:45:58.855979 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:45:58.855979 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:45:58.855979 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:45:58.855979 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:45:58.855979 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:45:58.855979 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:45:58.855979 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:45:58.855979 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:45:58.855979 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:45:58.855979 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:45:58.857195 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:45:58.857195 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:45:58.857195 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:45:58.857195 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:45:58.857195 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:45:58.857195 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:45:58.857195 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:45:58.857195 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:45:58.857195 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:45:58.857195 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:45:58.857195 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:45:58.857195 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:45:58.857195 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:45:58.857195 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:45:58.857195 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:45:58.857195 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:45:58.857964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:45:58.857964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:45:58.857964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:45:58.857964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:45:58.857964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:45:58.857964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:45:58.857964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:45:58.857964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:45:58.857964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:45:58.857964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:45:58.857964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 04:45:58.857964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:45:58.860104 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 84/169] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jul 26 04:45:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:45:58.866782 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29018b00-c253-4345-9b77-aab17189350b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:58.945941 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8fb2d-278a-404c-9714-7d2969989de5 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:59.044427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4728d4b-573e-4e73-8b6b-97f7afb4937a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:59.045012 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4728d4b-573e-4e73-8b6b-97f7afb4937a None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:59.045300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4728d4b-573e-4e73-8b6b-97f7afb4937a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:59.046743 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4728d4b-573e-4e73-8b6b-97f7afb4937a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:59.076705 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4728d4b-573e-4e73-8b6b-97f7afb4937a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:45:59.081883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-b4728d4b-573e-4e73-8b6b-97f7afb4937a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:45:59.081883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:45:59.081883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:45:59.081883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:45:59.081883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:45:59.081883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:45:59.081883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:45:59.081883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:45:59.081883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:45:59.081883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:45:59.081883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:45:59.081883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:45:59.081883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:45:59.081883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:45:59.081883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:45:59.081883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:45:59.082809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:45:59.082809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:45:59.082809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:45:59.082809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:45:59.082809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:45:59.082809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:45:59.082809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:45:59.082809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:45:59.082809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:45:59.082809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:45:59.082809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:45:59.082809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:45:59.082809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:45:59.082809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:45:59.082809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:45:59.082809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:45:59.083695 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:45:59.083695 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:45:59.083695 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:45:59.083695 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:45:59.083695 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:45:59.083695 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:45:59.083695 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:45:59.083695 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:45:59.083695 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:45:59.083695 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:45:59.083695 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:45:59.083695 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:45:59.088064 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 86/170] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:45:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:45:59.091274 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29018b00-c253-4345-9b77-aab17189350b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:59.093177 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4728d4b-573e-4e73-8b6b-97f7afb4937a None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:59.152019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:59.178195 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b272a7e8-dbd2-4604-9cea-7288875b76ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:59.178848 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272a7e8-dbd2-4604-9cea-7288875b76ea None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:59.179134 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272a7e8-dbd2-4604-9cea-7288875b76ea None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:59.179403 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272a7e8-dbd2-4604-9cea-7288875b76ea None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:59.180190 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b272a7e8-dbd2-4604-9cea-7288875b76ea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:59.180498 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b272a7e8-dbd2-4604-9cea-7288875b76ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:59.181913 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b272a7e8-dbd2-4604-9cea-7288875b76ea None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:59.199345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 85/171] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:45:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:59.205377 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b272a7e8-dbd2-4604-9cea-7288875b76ea None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:59.242687 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:59.271203 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4728d4b-573e-4e73-8b6b-97f7afb4937a None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:59.307297 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc178705-3b73-4511-ae28-39912eaa64fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:59.307886 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc178705-3b73-4511-ae28-39912eaa64fd None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:59.308171 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc178705-3b73-4511-ae28-39912eaa64fd None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:59.308449 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc178705-3b73-4511-ae28-39912eaa64fd None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:59.309192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc178705-3b73-4511-ae28-39912eaa64fd None admin] RBAC: Adding query filter params (name=member) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:45:59.309503 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc178705-3b73-4511-ae28-39912eaa64fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:59.310940 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc178705-3b73-4511-ae28-39912eaa64fd None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:59.324865 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 87/172] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:45:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:59.327453 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc178705-3b73-4511-ae28-39912eaa64fd None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:59.422495 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b272a7e8-dbd2-4604-9cea-7288875b76ea None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:59.498687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dab770-d556-4bfe-8523-00bd21084556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:59.499622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dab770-d556-4bfe-8523-00bd21084556 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:59.500064 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dab770-d556-4bfe-8523-00bd21084556 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:59.500378 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dab770-d556-4bfe-8523-00bd21084556 None admin] PATH_INFO: `/v3/users/7b1c7d019a3b49b79a0a9a3a4aea88f9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:59.503142 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dab770-d556-4bfe-8523-00bd21084556 None admin] RBAC: Authorizing `identity:get_user(user_id=7b1c7d019a3b49b79a0a9a3a4aea88f9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:59.504600 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dab770-d556-4bfe-8523-00bd21084556 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:59.506637 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 86/173] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:45:59 2022] GET /identity/v3/users/7b1c7d019a3b49b79a0a9a3a4aea88f9 => generated 275 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:45:59.512550 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dab770-d556-4bfe-8523-00bd21084556 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:59.526078 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc178705-3b73-4511-ae28-39912eaa64fd None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:59.564828 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c77d6-b92d-4394-aa84-5e29e0a36d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:59.565735 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c77d6-b92d-4394-aa84-5e29e0a36d0e None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:59.566162 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c77d6-b92d-4394-aa84-5e29e0a36d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:59.566585 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c77d6-b92d-4394-aa84-5e29e0a36d0e None admin] PATH_INFO: `/v3/users/7b1c7d019a3b49b79a0a9a3a4aea88f9` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:59.568070 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c77d6-b92d-4394-aa84-5e29e0a36d0e None admin] RBAC: Authorizing `identity:get_user(user_id=7b1c7d019a3b49b79a0a9a3a4aea88f9)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:59.569721 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c77d6-b92d-4394-aa84-5e29e0a36d0e None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:59.575678 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 88/174] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:45:59 2022] GET /identity/v3/users/7b1c7d019a3b49b79a0a9a3a4aea88f9 => generated 275 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:45:59.577809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c77d6-b92d-4394-aa84-5e29e0a36d0e None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:59.587881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:59.628894 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:59.680859 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dab770-d556-4bfe-8523-00bd21084556 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:59.721900 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9abaa1-74eb-4491-afb2-ed943d375e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:59.722563 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9abaa1-74eb-4491-afb2-ed943d375e6c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:59.722848 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9abaa1-74eb-4491-afb2-ed943d375e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:59.723125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9abaa1-74eb-4491-afb2-ed943d375e6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:59.737641 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c77d6-b92d-4394-aa84-5e29e0a36d0e None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:45:59.800372 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9abaa1-74eb-4491-afb2-ed943d375e6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:59.801881 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9abaa1-74eb-4491-afb2-ed943d375e6c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:45:59.864180 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:45:59.880333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e8753-3d49-4895-b93a-63b91bbe4a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:45:59.881005 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e8753-3d49-4895-b93a-63b91bbe4a6f None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:45:59.881293 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e8753-3d49-4895-b93a-63b91bbe4a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:45:59.881573 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e8753-3d49-4895-b93a-63b91bbe4a6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:45:59.912354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 87/175] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:45:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 04:45:59.918389 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9abaa1-74eb-4491-afb2-ed943d375e6c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:45:59.991862 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e8753-3d49-4895-b93a-63b91bbe4a6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:45:59.993401 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e8753-3d49-4895-b93a-63b91bbe4a6f None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:00.001860 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:00.027477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9abaa1-74eb-4491-afb2-ed943d375e6c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:00.065584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb481494-fe02-4631-a9db-e7a346f2e645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:00.066352 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb481494-fe02-4631-a9db-e7a346f2e645 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:00.066639 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb481494-fe02-4631-a9db-e7a346f2e645 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:00.066913 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb481494-fe02-4631-a9db-e7a346f2e645 None admin] PATH_INFO: `/v3/projects/8be3583b9359412ba4129e556beef77c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:00.072364 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb481494-fe02-4631-a9db-e7a346f2e645 None admin] RBAC: Authorizing `identity:get_project(project_id=8be3583b9359412ba4129e556beef77c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:00.073968 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb481494-fe02-4631-a9db-e7a346f2e645 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:00.080151 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 88/176] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:45:59 2022] GET /identity/v3/projects/8be3583b9359412ba4129e556beef77c => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:00.086346 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb481494-fe02-4631-a9db-e7a346f2e645 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:00.103798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 89/177] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:45:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 04:46:00.105184 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e8753-3d49-4895-b93a-63b91bbe4a6f None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:00.135973 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:00.207479 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e8753-3d49-4895-b93a-63b91bbe4a6f None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:00.215497 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:00.243630 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b66b4-600a-4c68-bfcb-a9f8d1c94e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:00.244389 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b66b4-600a-4c68-bfcb-a9f8d1c94e69 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:00.244584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb481494-fe02-4631-a9db-e7a346f2e645 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:00.244806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b66b4-600a-4c68-bfcb-a9f8d1c94e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:00.245163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b66b4-600a-4c68-bfcb-a9f8d1c94e69 None admin] PATH_INFO: `/v3/projects/6404c767e4bf43c8aa64c294fd5de203` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:00.246517 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b66b4-600a-4c68-bfcb-a9f8d1c94e69 None admin] RBAC: Authorizing `identity:get_project(project_id=6404c767e4bf43c8aa64c294fd5de203)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:00.248190 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b66b4-600a-4c68-bfcb-a9f8d1c94e69 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:00.254943 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 90/178] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:00 2022] GET /identity/v3/projects/6404c767e4bf43c8aa64c294fd5de203 => generated 312 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:00.256208 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b66b4-600a-4c68-bfcb-a9f8d1c94e69 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:00.300644 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d03f6-9260-4af0-8a9c-42aed3a2a2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:00.301765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d03f6-9260-4af0-8a9c-42aed3a2a2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:00.302238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d03f6-9260-4af0-8a9c-42aed3a2a2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:00.302723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d03f6-9260-4af0-8a9c-42aed3a2a2a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:00.303735 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d03f6-9260-4af0-8a9c-42aed3a2a2a9 None admin] RBAC: Adding query filter params (role.id=9223a9bb40484faca3a7bcfeb41a2215, scope.project.id=8be3583b9359412ba4129e556beef77c, user.id=7b1c7d019a3b49b79a0a9a3a4aea88f9) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:00.304251 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d03f6-9260-4af0-8a9c-42aed3a2a2a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:00.306057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d03f6-9260-4af0-8a9c-42aed3a2a2a9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:00.322829 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 89/179] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:46:00 2022] GET /identity/v3/role_assignments?user.id=7b1c7d019a3b49b79a0a9a3a4aea88f9&scope.project.id=8be3583b9359412ba4129e556beef77c&role.id=9223a9bb40484faca3a7bcfeb41a2215 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:00.370073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b66b4-600a-4c68-bfcb-a9f8d1c94e69 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:00.419616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d6029-c535-41f3-a1e4-158aa615755c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:00.420256 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d6029-c535-41f3-a1e4-158aa615755c None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:00.420542 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d6029-c535-41f3-a1e4-158aa615755c None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:00.420818 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d6029-c535-41f3-a1e4-158aa615755c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:00.421702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d6029-c535-41f3-a1e4-158aa615755c None admin] RBAC: Adding query filter params (role.id=7519356859fb40508c25bec8161207fd, scope.project.id=6404c767e4bf43c8aa64c294fd5de203, user.id=7b1c7d019a3b49b79a0a9a3a4aea88f9) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:00.422019 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d6029-c535-41f3-a1e4-158aa615755c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:00.423538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d6029-c535-41f3-a1e4-158aa615755c None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:00.433847 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 91/180] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:46:00 2022] GET /identity/v3/role_assignments?user.id=7b1c7d019a3b49b79a0a9a3a4aea88f9&scope.project.id=6404c767e4bf43c8aa64c294fd5de203&role.id=7519356859fb40508c25bec8161207fd => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:00.574193 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0eed9b-fee8-4d5b-b44c-fc044b185e04 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:00.575235 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0eed9b-fee8-4d5b-b44c-fc044b185e04 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:00.575767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0eed9b-fee8-4d5b-b44c-fc044b185e04 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:00.577688 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 90/181] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:00.583824 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa8e45-94be-4ed7-918a-29e55491153f None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:00.584323 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa8e45-94be-4ed7-918a-29e55491153f None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:00.584417 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa8e45-94be-4ed7-918a-29e55491153f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:00.613122 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-88fa8e45-94be-4ed7-918a-29e55491153f None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:00.614373 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-88fa8e45-94be-4ed7-918a-29e55491153f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:00.648953 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:00.691486 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 92/182] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:00.714901 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0eed9b-fee8-4d5b-b44c-fc044b185e04 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:00.724916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3192a7b-225a-419a-8c72-8384719ebf0f None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:00.725247 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3192a7b-225a-419a-8c72-8384719ebf0f None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:00.725481 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3192a7b-225a-419a-8c72-8384719ebf0f None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:00.726899 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 93/183] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:46:00.733255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678e3c1-8d45-4403-af39-906c197f735a None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:00.733577 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678e3c1-8d45-4403-af39-906c197f735a None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:00.733859 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678e3c1-8d45-4403-af39-906c197f735a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:00.757902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-1678e3c1-8d45-4403-af39-906c197f735a None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:00.759083 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-1678e3c1-8d45-4403-af39-906c197f735a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:00.827958 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:00.835534 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 94/184] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:00.847694 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0eed9b-fee8-4d5b-b44c-fc044b185e04 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:00.860823 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678e3c1-8d45-4403-af39-906c197f735a None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:00.887972 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b5977-934e-4c85-9b1e-dc5143402a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:00.889053 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b5977-934e-4c85-9b1e-dc5143402a3a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:00.889642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b5977-934e-4c85-9b1e-dc5143402a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:00.890180 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b5977-934e-4c85-9b1e-dc5143402a3a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:00.901975 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b5977-934e-4c85-9b1e-dc5143402a3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:00.908253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-368b5977-934e-4c85-9b1e-dc5143402a3a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 04:46:00.908253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:00.908253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:00.908253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:00.908253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:00.908253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:00.908253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:00.908253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:00.908253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:00.908253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:00.908253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:00.908253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:00.908253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:00.908253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:00.908253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:00.908253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:00.909152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:00.909152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:00.909152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:00.909152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:00.909152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:00.909152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:00.909152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:00.909152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:00.909152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:00.909152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:00.909152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:00.909152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:00.909152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:00.909152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:00.909152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:00.909152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:00.909923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:00.909923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:00.909923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:00.909923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:00.909923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:00.909923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:00.909923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:00.909923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:00.909923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:00.909923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:00.909923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 04:46:00.909923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:00.912484 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 91/185] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jul 26 04:46:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:00.918690 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b5977-934e-4c85-9b1e-dc5143402a3a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:00.962292 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:00.965949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678e3c1-8d45-4403-af39-906c197f735a None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:01.009739 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128119c-2b33-4014-bd24-48d54967b4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:01.010855 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128119c-2b33-4014-bd24-48d54967b4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:01.010855 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128119c-2b33-4014-bd24-48d54967b4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:01.011045 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128119c-2b33-4014-bd24-48d54967b4c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:01.022439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4128119c-2b33-4014-bd24-48d54967b4c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:01.027399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-4128119c-2b33-4014-bd24-48d54967b4c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:46:01.027399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:01.027399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:01.027399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:01.027399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:01.027399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:01.027399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:01.027399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:01.027399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:01.027399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:01.027399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:01.027399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:01.027399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:01.027399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:01.027399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:01.027399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:01.028333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:01.028333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:01.028333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:01.028333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:01.028333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:01.028333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:01.028333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:01.028333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:01.028333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:01.028333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:01.028333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:01.028333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:01.028333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:01.028333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:01.028333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:01.028333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:01.029098 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:01.029098 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:01.029098 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:01.029098 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:01.029098 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:01.029098 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:01.029098 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:01.029098 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:01.029098 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:01.029098 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:01.029098 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:46:01.029098 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:01.033672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 95/186] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:46:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:01.034971 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128119c-2b33-4014-bd24-48d54967b4c2 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:01.042093 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b5977-934e-4c85-9b1e-dc5143402a3a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:01.079872 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90045474-56ef-447e-8713-95a035d93eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:01.080971 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90045474-56ef-447e-8713-95a035d93eae None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:01.081708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90045474-56ef-447e-8713-95a035d93eae None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:01.082199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90045474-56ef-447e-8713-95a035d93eae None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:01.083180 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90045474-56ef-447e-8713-95a035d93eae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:01.083705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90045474-56ef-447e-8713-95a035d93eae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:01.085540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90045474-56ef-447e-8713-95a035d93eae None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:01.097187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 92/187] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:46:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:01.103443 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90045474-56ef-447e-8713-95a035d93eae None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:01.123946 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:01.142882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128119c-2b33-4014-bd24-48d54967b4c2 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:01.181968 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c4bb1-2372-43dd-84a4-b6dfa0cfcb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:01.182565 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c4bb1-2372-43dd-84a4-b6dfa0cfcb20 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:01.182783 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c4bb1-2372-43dd-84a4-b6dfa0cfcb20 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:01.183052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c4bb1-2372-43dd-84a4-b6dfa0cfcb20 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:01.183815 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9c4bb1-2372-43dd-84a4-b6dfa0cfcb20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:01.184115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9c4bb1-2372-43dd-84a4-b6dfa0cfcb20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:01.185515 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9c4bb1-2372-43dd-84a4-b6dfa0cfcb20 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:01.194872 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 96/188] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:01.196209 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff60099-c101-42c9-b748-925cf225c32c None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:01.196524 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff60099-c101-42c9-b748-925cf225c32c None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:01.196798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff60099-c101-42c9-b748-925cf225c32c None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:01.198149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 97/189] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:01.200925 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff60099-c101-42c9-b748-925cf225c32c None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:01.227949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:01.233601 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90045474-56ef-447e-8713-95a035d93eae None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:01.274536 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b4c031-033c-427d-b868-3f3aba6d0f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:01.275329 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4c031-033c-427d-b868-3f3aba6d0f0b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:01.275642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4c031-033c-427d-b868-3f3aba6d0f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:01.275924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4c031-033c-427d-b868-3f3aba6d0f0b None admin] PATH_INFO: `/v3/users/0773b663533d44ba8d001f743edcce45` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:01.291511 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b4c031-033c-427d-b868-3f3aba6d0f0b None admin] RBAC: Authorizing `identity:get_user(user_id=0773b663533d44ba8d001f743edcce45)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:01.293169 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b4c031-033c-427d-b868-3f3aba6d0f0b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:01.299641 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 93/190] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:46:01 2022] GET /identity/v3/users/0773b663533d44ba8d001f743edcce45 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:01.299641 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5ae10-36e4-416b-8245-430c430c66e7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:01.299641 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5ae10-36e4-416b-8245-430c430c66e7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:01.299641 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5ae10-36e4-416b-8245-430c430c66e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:01.316285 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b4a5ae10-36e4-416b-8245-430c430c66e7 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:01.317205 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff60099-c101-42c9-b748-925cf225c32c None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:01.317494 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b4a5ae10-36e4-416b-8245-430c430c66e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:01.357577 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ba20b-38e6-41a7-8830-e487765dd1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:01.358237 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ba20b-38e6-41a7-8830-e487765dd1f8 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:01.358552 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ba20b-38e6-41a7-8830-e487765dd1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:01.358837 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ba20b-38e6-41a7-8830-e487765dd1f8 None admin] PATH_INFO: `/v3/users/0773b663533d44ba8d001f743edcce45` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:01.360052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228ba20b-38e6-41a7-8830-e487765dd1f8 None admin] RBAC: Authorizing `identity:get_user(user_id=0773b663533d44ba8d001f743edcce45)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:01.361515 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228ba20b-38e6-41a7-8830-e487765dd1f8 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:01.368005 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 98/191] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:46:01 2022] GET /identity/v3/users/0773b663533d44ba8d001f743edcce45 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:01.368907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ba20b-38e6-41a7-8830-e487765dd1f8 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:01.393843 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:01.402888 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 94/192] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:01.411738 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5ae10-36e4-416b-8245-430c430c66e7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:01.521160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5ae10-36e4-416b-8245-430c430c66e7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:01.525560 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:01.547410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ba20b-38e6-41a7-8830-e487765dd1f8 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:01.559657 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58c346-09d1-4cd9-87b0-f399fe99a7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:01.560342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58c346-09d1-4cd9-87b0-f399fe99a7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:01.560621 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58c346-09d1-4cd9-87b0-f399fe99a7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:01.560899 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58c346-09d1-4cd9-87b0-f399fe99a7e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:01.606057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd49f45-23af-4106-b5f4-d99db35e2d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:01.607144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd49f45-23af-4106-b5f4-d99db35e2d7b None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:01.607687 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd49f45-23af-4106-b5f4-d99db35e2d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:01.608173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd49f45-23af-4106-b5f4-d99db35e2d7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:01.628387 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:01.637304 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb58c346-09d1-4cd9-87b0-f399fe99a7e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:01.638881 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb58c346-09d1-4cd9-87b0-f399fe99a7e4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:01.686700 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd49f45-23af-4106-b5f4-d99db35e2d7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:01.688200 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd49f45-23af-4106-b5f4-d99db35e2d7b None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:01.742480 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 95/193] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:46:01.743352 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58c346-09d1-4cd9-87b0-f399fe99a7e4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:01.792369 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:01.810255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 99/194] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:46:01.811807 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c571b-6238-4e81-92d0-50b94a955e59 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:01.812132 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c571b-6238-4e81-92d0-50b94a955e59 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:01.812434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c571b-6238-4e81-92d0-50b94a955e59 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:01.814033 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 100/195] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:01.814761 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c571b-6238-4e81-92d0-50b94a955e59 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:01.858746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:01.865385 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58c346-09d1-4cd9-87b0-f399fe99a7e4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:01.901901 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd0c5d-26fc-4a4f-a2e3-57e9f67c2b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:01.902562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd0c5d-26fc-4a4f-a2e3-57e9f67c2b31 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:01.902850 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd0c5d-26fc-4a4f-a2e3-57e9f67c2b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:01.903126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd0c5d-26fc-4a4f-a2e3-57e9f67c2b31 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:01.904429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd0c5d-26fc-4a4f-a2e3-57e9f67c2b31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:01.905921 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd0c5d-26fc-4a4f-a2e3-57e9f67c2b31 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:01.907654 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 96/196] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:01.908419 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd0c5d-26fc-4a4f-a2e3-57e9f67c2b31 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:01.926824 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c571b-6238-4e81-92d0-50b94a955e59 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:01.962033 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d27f1a7-a919-40e1-a50c-d471e237033a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:01.962670 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27f1a7-a919-40e1-a50c-d471e237033a None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:01.962946 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27f1a7-a919-40e1-a50c-d471e237033a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:01.963212 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27f1a7-a919-40e1-a50c-d471e237033a None admin] PATH_INFO: `/v3/projects/fce7a22bc8ff438d90035074a2c73590` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:01.964404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d27f1a7-a919-40e1-a50c-d471e237033a None admin] RBAC: Authorizing `identity:get_project(project_id=fce7a22bc8ff438d90035074a2c73590)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:01.965860 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d27f1a7-a919-40e1-a50c-d471e237033a None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:01.967582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 101/197] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:01 2022] GET /identity/v3/projects/fce7a22bc8ff438d90035074a2c73590 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:01.968799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c1733-7797-4d12-a793-addeff3b43e5 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:01.969071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c1733-7797-4d12-a793-addeff3b43e5 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:01.969359 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c1733-7797-4d12-a793-addeff3b43e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:01.988222 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-385c1733-7797-4d12-a793-addeff3b43e5 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:01.989444 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-385c1733-7797-4d12-a793-addeff3b43e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:02.025009 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd0c5d-26fc-4a4f-a2e3-57e9f67c2b31 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:02.047219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:02.066345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbdd0f7-54ef-4c9f-acff-9de9a4b874ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:02.066345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdd0f7-54ef-4c9f-acff-9de9a4b874ac None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:02.066345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdd0f7-54ef-4c9f-acff-9de9a4b874ac None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:02.066743 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdd0f7-54ef-4c9f-acff-9de9a4b874ac None admin] PATH_INFO: `/v3/projects/fce7a22bc8ff438d90035074a2c73590` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:02.067403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 102/198] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:02.074810 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbdd0f7-54ef-4c9f-acff-9de9a4b874ac None admin] RBAC: Authorizing `identity:get_project(project_id=fce7a22bc8ff438d90035074a2c73590)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:02.074899 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c1733-7797-4d12-a793-addeff3b43e5 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:02.074969 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbdd0f7-54ef-4c9f-acff-9de9a4b874ac None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:02.074969 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 97/199] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:01 2022] GET /identity/v3/projects/fce7a22bc8ff438d90035074a2c73590 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:02.074969 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbdd0f7-54ef-4c9f-acff-9de9a4b874ac None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:02.116363 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:02.177231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c1733-7797-4d12-a793-addeff3b43e5 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:02.185544 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbdd0f7-54ef-4c9f-acff-9de9a4b874ac None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:02.211489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869f183-08fa-4e5e-94ce-c0b5041bf92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:02.212190 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869f183-08fa-4e5e-94ce-c0b5041bf92d None admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:02.212439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869f183-08fa-4e5e-94ce-c0b5041bf92d None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:02.212706 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869f183-08fa-4e5e-94ce-c0b5041bf92d None admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:02.213398 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869f183-08fa-4e5e-94ce-c0b5041bf92d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:02.214875 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869f183-08fa-4e5e-94ce-c0b5041bf92d None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:02.221837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd27470-12a1-488d-a01f-f226e5466203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:02.222559 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd27470-12a1-488d-a01f-f226e5466203 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:02.222821 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd27470-12a1-488d-a01f-f226e5466203 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:02.223233 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd27470-12a1-488d-a01f-f226e5466203 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:02.224103 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd27470-12a1-488d-a01f-f226e5466203 None admin] RBAC: Adding query filter params (role.id=72bf8bd23d2c4b76ad32b7b35d6db412, scope.project.id=fce7a22bc8ff438d90035074a2c73590, user.id=0773b663533d44ba8d001f743edcce45) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:02.224413 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd27470-12a1-488d-a01f-f226e5466203 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:02.225858 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd27470-12a1-488d-a01f-f226e5466203 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:02.232974 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 98/200] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:46:02 2022] GET /identity/v3/role_assignments?user.id=0773b663533d44ba8d001f743edcce45&scope.project.id=fce7a22bc8ff438d90035074a2c73590&role.id=72bf8bd23d2c4b76ad32b7b35d6db412 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:02.233822 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd27470-12a1-488d-a01f-f226e5466203 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:02.251770 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 103/201] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jul 26 04:46:02 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:46:02.253161 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869f183-08fa-4e5e-94ce-c0b5041bf92d None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:02.320130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:02.364077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:02.371831 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd27470-12a1-488d-a01f-f226e5466203 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:02.395013 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869f183-08fa-4e5e-94ce-c0b5041bf92d None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:02.416931 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4399bec5-cfa1-4d9b-8842-e2336d44dc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:02.417686 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399bec5-cfa1-4d9b-8842-e2336d44dc28 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:02.418034 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399bec5-cfa1-4d9b-8842-e2336d44dc28 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:02.418378 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399bec5-cfa1-4d9b-8842-e2336d44dc28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:02.419452 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4399bec5-cfa1-4d9b-8842-e2336d44dc28 None admin] RBAC: Adding query filter params (role.id=9223a9bb40484faca3a7bcfeb41a2215, scope.project.id=fce7a22bc8ff438d90035074a2c73590, user.id=0773b663533d44ba8d001f743edcce45) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:02.419875 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4399bec5-cfa1-4d9b-8842-e2336d44dc28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:02.423025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4399bec5-cfa1-4d9b-8842-e2336d44dc28 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:02.431969 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 99/202] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:46:02 2022] GET /identity/v3/role_assignments?user.id=0773b663533d44ba8d001f743edcce45&scope.project.id=fce7a22bc8ff438d90035074a2c73590&role.id=9223a9bb40484faca3a7bcfeb41a2215 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:02.437024 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c880c1-6c73-4f8f-b537-9a8b10579248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:02.437723 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c880c1-6c73-4f8f-b537-9a8b10579248 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:02.438007 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c880c1-6c73-4f8f-b537-9a8b10579248 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:02.438283 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c880c1-6c73-4f8f-b537-9a8b10579248 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:02.452343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c880c1-6c73-4f8f-b537-9a8b10579248 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:02.458245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-f6c880c1-6c73-4f8f-b537-9a8b10579248 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 04:46:02.458245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:02.458245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:02.458245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:02.458245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:02.458245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:02.458245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:02.458245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:02.458245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:02.458245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:02.458245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:02.458245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:02.458245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:02.458245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:02.458245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:02.458245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:02.460105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:02.460105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:02.460105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:02.460105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:02.460105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:02.460105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:02.460105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:02.460105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:02.460105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:02.460105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:02.460105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:02.460105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:02.460105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:02.460105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:02.460105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:02.460105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:02.461642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:02.461642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:02.461642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:02.461642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:02.461642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:02.461642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:02.461642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:02.461642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:02.461642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:02.461642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:02.461642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 04:46:02.461642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:02.461642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 104/203] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:02.467065 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4399bec5-cfa1-4d9b-8842-e2336d44dc28 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:02.649798 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4399bec5-cfa1-4d9b-8842-e2336d44dc28 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:02.695811 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6965ef-c2fa-4ea1-aec8-fb0abe0fe033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:02.696777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6965ef-c2fa-4ea1-aec8-fb0abe0fe033 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:02.697237 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6965ef-c2fa-4ea1-aec8-fb0abe0fe033 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:02.697737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6965ef-c2fa-4ea1-aec8-fb0abe0fe033 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:02.699008 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6965ef-c2fa-4ea1-aec8-fb0abe0fe033 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:02.699540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6965ef-c2fa-4ea1-aec8-fb0abe0fe033 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:02.701111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6965ef-c2fa-4ea1-aec8-fb0abe0fe033 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:02.715393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 100/204] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:46:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:02.721885 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c880c1-6c73-4f8f-b537-9a8b10579248 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:02.860004 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c880c1-6c73-4f8f-b537-9a8b10579248 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:02.861638 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:02.910014 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a374c1-8e3f-4a5e-a397-44f705de5953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:02.910598 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a374c1-8e3f-4a5e-a397-44f705de5953 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:02.910885 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a374c1-8e3f-4a5e-a397-44f705de5953 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:02.911162 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a374c1-8e3f-4a5e-a397-44f705de5953 None admin] PATH_INFO: `/v3/users/14fe00abc013460abd04ea1b5f2ed9ee` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:02.916501 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a374c1-8e3f-4a5e-a397-44f705de5953 None admin] RBAC: Authorizing `identity:get_user(user_id=14fe00abc013460abd04ea1b5f2ed9ee)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:02.918031 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a374c1-8e3f-4a5e-a397-44f705de5953 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:02.920083 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 105/205] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:46:02 2022] GET /identity/v3/users/14fe00abc013460abd04ea1b5f2ed9ee => generated 289 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:02.925994 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6965ef-c2fa-4ea1-aec8-fb0abe0fe033 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:02.962159 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:03.063213 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6965ef-c2fa-4ea1-aec8-fb0abe0fe033 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:03.110111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6f888-8513-4420-8638-da9d9ac2144e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:03.110758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6f888-8513-4420-8638-da9d9ac2144e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:03.111041 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6f888-8513-4420-8638-da9d9ac2144e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:03.111317 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6f888-8513-4420-8638-da9d9ac2144e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:03.192799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f6f888-8513-4420-8638-da9d9ac2144e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:03.194276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f6f888-8513-4420-8638-da9d9ac2144e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:03.219481 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:03.332283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 101/206] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:46:03.338290 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a374c1-8e3f-4a5e-a397-44f705de5953 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:03.461868 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a374c1-8e3f-4a5e-a397-44f705de5953 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:03.495016 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a648c-9b49-4cba-8b7d-0b40cad5c5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:03.495703 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a648c-9b49-4cba-8b7d-0b40cad5c5dc None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:03.495986 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a648c-9b49-4cba-8b7d-0b40cad5c5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:03.496265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a648c-9b49-4cba-8b7d-0b40cad5c5dc None admin] PATH_INFO: `/v3/projects/8be3583b9359412ba4129e556beef77c` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:03.497442 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a648c-9b49-4cba-8b7d-0b40cad5c5dc None admin] RBAC: Authorizing `identity:get_project(project_id=8be3583b9359412ba4129e556beef77c)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:03.498986 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a648c-9b49-4cba-8b7d-0b40cad5c5dc None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:03.501114 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 106/207] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:03 2022] GET /identity/v3/projects/8be3583b9359412ba4129e556beef77c => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:03.508719 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6f888-8513-4420-8638-da9d9ac2144e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:03.640540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6f888-8513-4420-8638-da9d9ac2144e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:03.683060 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:03.697336 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1fd92-588e-4e5f-85c0-37cca1db43bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:03.698248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1fd92-588e-4e5f-85c0-37cca1db43bb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:03.698695 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1fd92-588e-4e5f-85c0-37cca1db43bb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:03.699152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1fd92-588e-4e5f-85c0-37cca1db43bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:03.700615 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1fd92-588e-4e5f-85c0-37cca1db43bb None admin] RBAC: Adding query filter params (role.id=6c31395d654e4add9c982f62b79c2873, scope.project.id=8be3583b9359412ba4129e556beef77c, user.id=14fe00abc013460abd04ea1b5f2ed9ee) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:03.701141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1fd92-588e-4e5f-85c0-37cca1db43bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:03.703840 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1fd92-588e-4e5f-85c0-37cca1db43bb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:03.716953 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 102/208] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:46:03 2022] GET /identity/v3/role_assignments?user.id=14fe00abc013460abd04ea1b5f2ed9ee&scope.project.id=8be3583b9359412ba4129e556beef77c&role.id=6c31395d654e4add9c982f62b79c2873 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:04.667497 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a2e79-9258-4ffa-8e49-094dadf05635 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:04.668497 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a2e79-9258-4ffa-8e49-094dadf05635 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:04.668968 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a2e79-9258-4ffa-8e49-094dadf05635 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:04.670900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 107/209] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:04.676941 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ac762-9067-4a29-b33e-d7aa10aab2e8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:04.677483 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ac762-9067-4a29-b33e-d7aa10aab2e8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:04.677948 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ac762-9067-4a29-b33e-d7aa10aab2e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:04.698221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-274ac762-9067-4a29-b33e-d7aa10aab2e8 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:04.700237 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-274ac762-9067-4a29-b33e-d7aa10aab2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:04.794840 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 103/210] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 04:46:04.817497 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a2e79-9258-4ffa-8e49-094dadf05635 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:04.861671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:04.926281 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a2e79-9258-4ffa-8e49-094dadf05635 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:04.952301 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e28dd-8956-4b37-a742-7a5a2cdbe37b None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:04.952634 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e28dd-8956-4b37-a742-7a5a2cdbe37b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:04.952869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e28dd-8956-4b37-a742-7a5a2cdbe37b None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:04.954565 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 104/211] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:04.960031 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10cb47-0f43-401d-9638-ba395c2be9cd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:04.960319 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10cb47-0f43-401d-9638-ba395c2be9cd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:04.960597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10cb47-0f43-401d-9638-ba395c2be9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:04.963775 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19570a-bc7a-41e5-9fe4-efac18d00436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:04.964439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19570a-bc7a-41e5-9fe4-efac18d00436 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:04.964709 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19570a-bc7a-41e5-9fe4-efac18d00436 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:04.964975 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19570a-bc7a-41e5-9fe4-efac18d00436 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:04.977185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e19570a-bc7a-41e5-9fe4-efac18d00436 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:04.981082 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8c10cb47-0f43-401d-9638-ba395c2be9cd None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:04.982158 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-5e19570a-bc7a-41e5-9fe4-efac18d00436 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:46:04.982158 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:04.982158 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:04.982158 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:04.982158 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:04.982158 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:04.982158 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:04.982158 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:04.982158 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:04.982158 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:04.982158 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:04.982158 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:04.982158 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:04.982158 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:04.982158 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:04.982158 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:04.983035 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:04.983035 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:04.983035 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:04.983035 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:04.983035 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:04.983035 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:04.983035 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:04.983035 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:04.983035 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:04.983035 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:04.983035 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:04.983035 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:04.983035 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:04.983035 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:04.983035 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:04.983035 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:04.984244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:04.984244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:04.984244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:04.984244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:04.984244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:04.984244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:04.984244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:04.984244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:04.984244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:04.984244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:04.984244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:46:04.984244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:04.985536 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8c10cb47-0f43-401d-9638-ba395c2be9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:04.987667 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 108/212] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:46:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:04.989667 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19570a-bc7a-41e5-9fe4-efac18d00436 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:05.072209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 105/213] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:05.077400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82226f0-bc8a-4fed-b896-31ab261e908f None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:05.078083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82226f0-bc8a-4fed-b896-31ab261e908f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:05.078559 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82226f0-bc8a-4fed-b896-31ab261e908f None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:05.080335 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 106/214] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:46:05.086539 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168df77d-6760-4198-ab25-4f1f5fff29db None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:05.086840 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168df77d-6760-4198-ab25-4f1f5fff29db None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:05.087458 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168df77d-6760-4198-ab25-4f1f5fff29db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:05.096134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:05.101597 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19570a-bc7a-41e5-9fe4-efac18d00436 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:05.109166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-168df77d-6760-4198-ab25-4f1f5fff29db None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:05.110472 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-168df77d-6760-4198-ab25-4f1f5fff29db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:05.142235 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59d2ff-8c5c-4d45-8ece-ec91d4c2d064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:05.142870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59d2ff-8c5c-4d45-8ece-ec91d4c2d064 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:05.143144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59d2ff-8c5c-4d45-8ece-ec91d4c2d064 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:05.143413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59d2ff-8c5c-4d45-8ece-ec91d4c2d064 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:05.144172 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d59d2ff-8c5c-4d45-8ece-ec91d4c2d064 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:05.144471 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d59d2ff-8c5c-4d45-8ece-ec91d4c2d064 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:05.145881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d59d2ff-8c5c-4d45-8ece-ec91d4c2d064 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:05.154396 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 109/215] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:05.156586 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59d2ff-8c5c-4d45-8ece-ec91d4c2d064 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:05.165180 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:05.193072 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 107/216] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:05.194422 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96baee3f-fe1d-4b08-8a37-9610cfd6f4a9 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:05.194713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96baee3f-fe1d-4b08-8a37-9610cfd6f4a9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:05.194984 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96baee3f-fe1d-4b08-8a37-9610cfd6f4a9 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:05.196428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 108/217] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:05.197036 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96baee3f-fe1d-4b08-8a37-9610cfd6f4a9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:05.267794 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59d2ff-8c5c-4d45-8ece-ec91d4c2d064 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:05.302728 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96925cd7-12cd-4e5a-8f4e-4ea2720ca78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:05.303375 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96925cd7-12cd-4e5a-8f4e-4ea2720ca78e None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:05.303689 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96925cd7-12cd-4e5a-8f4e-4ea2720ca78e None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:05.303962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96925cd7-12cd-4e5a-8f4e-4ea2720ca78e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:05.305855 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96baee3f-fe1d-4b08-8a37-9610cfd6f4a9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:05.314450 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96925cd7-12cd-4e5a-8f4e-4ea2720ca78e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:05.319843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-96925cd7-12cd-4e5a-8f4e-4ea2720ca78e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 04:46:05.319843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:05.319843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:05.319843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:05.319843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:05.319843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:05.319843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:05.319843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:05.319843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:05.319843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:05.319843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:05.319843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:05.319843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:05.319843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:05.319843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:05.319843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:05.320765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:05.320765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:05.320765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:05.320765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:05.320765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:05.320765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:05.320765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:05.320765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:05.320765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:05.320765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:05.320765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:05.320765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:05.320765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:05.320765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:05.320765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:05.320765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:05.321590 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:05.321590 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:05.321590 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:05.321590 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:05.321590 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:05.321590 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:05.321590 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:05.321590 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:05.321590 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:05.321590 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:05.321590 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 04:46:05.321590 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:05.321590 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 110/218] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jul 26 04:46:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:05.322315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffd785-ccc4-4a8d-bec2-49e37d3da2f4 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:05.322559 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffd785-ccc4-4a8d-bec2-49e37d3da2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:05.322766 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffd785-ccc4-4a8d-bec2-49e37d3da2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:05.343114 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-5cffd785-ccc4-4a8d-bec2-49e37d3da2f4 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:05.344386 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-5cffd785-ccc4-4a8d-bec2-49e37d3da2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:05.348143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d825fcb-f91a-48bf-8057-189c7e1fafbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:05.348842 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d825fcb-f91a-48bf-8057-189c7e1fafbf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:05.349132 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d825fcb-f91a-48bf-8057-189c7e1fafbf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:05.349408 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d825fcb-f91a-48bf-8057-189c7e1fafbf None admin] PATH_INFO: `/v3/users/0773b663533d44ba8d001f743edcce45` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:05.350750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d825fcb-f91a-48bf-8057-189c7e1fafbf None admin] RBAC: Authorizing `identity:get_user(user_id=0773b663533d44ba8d001f743edcce45)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:05.352291 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d825fcb-f91a-48bf-8057-189c7e1fafbf None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:05.354049 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 109/219] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:46:05 2022] GET /identity/v3/users/0773b663533d44ba8d001f743edcce45 => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:05.354887 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d825fcb-f91a-48bf-8057-189c7e1fafbf None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:05.366499 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:05.408974 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:05.431858 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 111/220] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:05.432844 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cffd785-ccc4-4a8d-bec2-49e37d3da2f4 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:05.482685 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d825fcb-f91a-48bf-8057-189c7e1fafbf None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:05.519852 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e21d9-1dfb-49a6-aad2-5713c567d2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:05.520520 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e21d9-1dfb-49a6-aad2-5713c567d2ce None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:05.520806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e21d9-1dfb-49a6-aad2-5713c567d2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:05.521085 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e21d9-1dfb-49a6-aad2-5713c567d2ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:05.531363 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e21d9-1dfb-49a6-aad2-5713c567d2ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:05.536298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-da9e21d9-1dfb-49a6-aad2-5713c567d2ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:46:05.536298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:05.536298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:05.536298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:05.536298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:05.536298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:05.536298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:05.536298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:05.536298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:05.536298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:05.536298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:05.536298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:05.536298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:05.536298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:05.536298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:05.536298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:05.537238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:05.537238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:05.537238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:05.537238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:05.537238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:05.537238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:05.537238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:05.537238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:05.537238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:05.537238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:05.537238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:05.537238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:05.537238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:05.537238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:05.537238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:05.537238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:05.538204 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:05.538204 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:05.538204 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:05.538204 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:05.538204 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:05.538204 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:05.538204 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:05.538204 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:05.538204 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:05.538204 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:05.538204 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:46:05.538204 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:05.538204 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 110/221] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:05.538988 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e21d9-1dfb-49a6-aad2-5713c567d2ce None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:05.569069 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cffd785-ccc4-4a8d-bec2-49e37d3da2f4 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:05.605443 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdfa37-8152-4258-831c-0285e471b5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:05.606067 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdfa37-8152-4258-831c-0285e471b5fa None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:05.606359 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdfa37-8152-4258-831c-0285e471b5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:05.606634 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdfa37-8152-4258-831c-0285e471b5fa None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:05.607367 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cdfa37-8152-4258-831c-0285e471b5fa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:05.607722 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cdfa37-8152-4258-831c-0285e471b5fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:05.609236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cdfa37-8152-4258-831c-0285e471b5fa None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:05.621159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 112/222] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:46:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:05.621986 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdfa37-8152-4258-831c-0285e471b5fa None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:05.638086 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:05.641861 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e21d9-1dfb-49a6-aad2-5713c567d2ce None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:05.644896 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:05.677281 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90732951-6db7-4057-acd4-3e7959fc3b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:05.677942 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90732951-6db7-4057-acd4-3e7959fc3b3c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:05.678228 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90732951-6db7-4057-acd4-3e7959fc3b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:05.678506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90732951-6db7-4057-acd4-3e7959fc3b3c None admin] PATH_INFO: `/v3/projects/fce7a22bc8ff438d90035074a2c73590` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:05.679794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90732951-6db7-4057-acd4-3e7959fc3b3c None admin] RBAC: Authorizing `identity:get_project(project_id=fce7a22bc8ff438d90035074a2c73590)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:05.681264 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90732951-6db7-4057-acd4-3e7959fc3b3c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:05.683180 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 111/223] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:05 2022] GET /identity/v3/projects/fce7a22bc8ff438d90035074a2c73590 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:05.683804 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90732951-6db7-4057-acd4-3e7959fc3b3c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:05.737393 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdfa37-8152-4258-831c-0285e471b5fa None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:05.772115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b6cd2-a98a-45f3-aa9c-53600d2fd003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:05.772797 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b6cd2-a98a-45f3-aa9c-53600d2fd003 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:05.773076 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b6cd2-a98a-45f3-aa9c-53600d2fd003 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:05.773350 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b6cd2-a98a-45f3-aa9c-53600d2fd003 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:05.774533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b6cd2-a98a-45f3-aa9c-53600d2fd003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:05.776069 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b6cd2-a98a-45f3-aa9c-53600d2fd003 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:05.777672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 113/224] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:05.778487 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b6cd2-a98a-45f3-aa9c-53600d2fd003 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:05.786379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90732951-6db7-4057-acd4-3e7959fc3b3c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:05.819989 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c31e9-eacc-4d11-bafa-a1b6eb80cbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:05.820618 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c31e9-eacc-4d11-bafa-a1b6eb80cbe9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:05.820899 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c31e9-eacc-4d11-bafa-a1b6eb80cbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:05.821172 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c31e9-eacc-4d11-bafa-a1b6eb80cbe9 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:05.821893 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c31e9-eacc-4d11-bafa-a1b6eb80cbe9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:05.822197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c31e9-eacc-4d11-bafa-a1b6eb80cbe9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:05.823651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c31e9-eacc-4d11-bafa-a1b6eb80cbe9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:05.833948 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 112/225] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:46:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:05.835667 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c31e9-eacc-4d11-bafa-a1b6eb80cbe9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:05.861234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:05.882869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:05.897396 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b6cd2-a98a-45f3-aa9c-53600d2fd003 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:05.938593 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3063bea-fe51-419f-b80b-d08dc121dd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:05.939239 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3063bea-fe51-419f-b80b-d08dc121dd78 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:05.939631 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3063bea-fe51-419f-b80b-d08dc121dd78 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:05.939907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3063bea-fe51-419f-b80b-d08dc121dd78 None admin] PATH_INFO: `/v3/users/0773b663533d44ba8d001f743edcce45` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:05.941103 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3063bea-fe51-419f-b80b-d08dc121dd78 None admin] RBAC: Authorizing `identity:get_user(user_id=0773b663533d44ba8d001f743edcce45)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:05.942577 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3063bea-fe51-419f-b80b-d08dc121dd78 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:05.944233 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 114/226] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:46:05 2022] GET /identity/v3/users/0773b663533d44ba8d001f743edcce45 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:05.944980 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3063bea-fe51-419f-b80b-d08dc121dd78 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:05.949422 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c31e9-eacc-4d11-bafa-a1b6eb80cbe9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:05.985946 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fe382-41e2-4fe5-8ca9-3a5169ee49ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:05.986629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fe382-41e2-4fe5-8ca9-3a5169ee49ca None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:05.986910 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fe382-41e2-4fe5-8ca9-3a5169ee49ca None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:05.987657 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fe382-41e2-4fe5-8ca9-3a5169ee49ca None admin] PATH_INFO: `/v3/projects/fce7a22bc8ff438d90035074a2c73590/users/0773b663533d44ba8d001f743edcce45/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:05.989860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fe382-41e2-4fe5-8ca9-3a5169ee49ca None admin] RBAC: Authorizing `identity:create_grant(project_id=fce7a22bc8ff438d90035074a2c73590, user_id=0773b663533d44ba8d001f743edcce45, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:05.991688 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fe382-41e2-4fe5-8ca9-3a5169ee49ca None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:06.005259 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 113/227] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jul 26 04:46:05 2022] PUT /identity/v3/projects/fce7a22bc8ff438d90035074a2c73590/users/0773b663533d44ba8d001f743edcce45/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:46:06.006103 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fe382-41e2-4fe5-8ca9-3a5169ee49ca None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:06.063722 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3063bea-fe51-419f-b80b-d08dc121dd78 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:06.105899 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e687192-470d-447d-82e8-6e60402ddd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:06.106580 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e687192-470d-447d-82e8-6e60402ddd4a None admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:06.106868 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e687192-470d-447d-82e8-6e60402ddd4a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:06.107144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e687192-470d-447d-82e8-6e60402ddd4a None admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:06.107904 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e687192-470d-447d-82e8-6e60402ddd4a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:06.109470 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e687192-470d-447d-82e8-6e60402ddd4a None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:06.115867 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:06.129651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fe382-41e2-4fe5-8ca9-3a5169ee49ca None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:06.143591 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 115/228] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jul 26 04:46:05 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:46:06.143591 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e687192-470d-447d-82e8-6e60402ddd4a None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:06.148045 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:06.187296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b924017-2aca-4eb8-8060-afe462963ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:06.188046 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b924017-2aca-4eb8-8060-afe462963ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:06.188332 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b924017-2aca-4eb8-8060-afe462963ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:06.188698 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b924017-2aca-4eb8-8060-afe462963ff9 None admin] PATH_INFO: `/v3/users/a6ab10409e8d4a1fa6350a506c28aa68` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:06.208723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b924017-2aca-4eb8-8060-afe462963ff9 None admin] RBAC: Authorizing `identity:get_user(user_id=a6ab10409e8d4a1fa6350a506c28aa68)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:06.210303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b924017-2aca-4eb8-8060-afe462963ff9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:06.212792 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 114/229] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:46:06 2022] GET /identity/v3/users/a6ab10409e8d4a1fa6350a506c28aa68 => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:06.218437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b924017-2aca-4eb8-8060-afe462963ff9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:06.297071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e687192-470d-447d-82e8-6e60402ddd4a None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:06.350630 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e51b4-579b-474c-a5af-8a0615e17c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:06.351854 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e51b4-579b-474c-a5af-8a0615e17c69 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:06.352277 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e51b4-579b-474c-a5af-8a0615e17c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:06.352848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e51b4-579b-474c-a5af-8a0615e17c69 None admin] PATH_INFO: `/v3/projects/fce7a22bc8ff438d90035074a2c73590` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:06.355314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e51b4-579b-474c-a5af-8a0615e17c69 None admin] RBAC: Authorizing `identity:get_project(project_id=fce7a22bc8ff438d90035074a2c73590)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:06.356837 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e51b4-579b-474c-a5af-8a0615e17c69 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:06.357520 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b924017-2aca-4eb8-8060-afe462963ff9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:06.358856 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 116/230] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:06 2022] GET /identity/v3/projects/fce7a22bc8ff438d90035074a2c73590 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:06.364478 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e51b4-579b-474c-a5af-8a0615e17c69 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:06.413304 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55518e-d356-4b9a-9926-db854a00d275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:06.414290 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55518e-d356-4b9a-9926-db854a00d275 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:06.414748 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55518e-d356-4b9a-9926-db854a00d275 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:06.415200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55518e-d356-4b9a-9926-db854a00d275 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:06.445438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:06.468792 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:06.511303 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e51b4-579b-474c-a5af-8a0615e17c69 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:06.511522 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f55518e-d356-4b9a-9926-db854a00d275 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:06.512949 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f55518e-d356-4b9a-9926-db854a00d275 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:06.554263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1f08c-2568-4fc4-bd18-6c7448fd469b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:06.554871 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1f08c-2568-4fc4-bd18-6c7448fd469b None admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:06.555149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1f08c-2568-4fc4-bd18-6c7448fd469b None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:06.555466 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1f08c-2568-4fc4-bd18-6c7448fd469b None admin] PATH_INFO: `/v3/projects/fce7a22bc8ff438d90035074a2c73590/users/0773b663533d44ba8d001f743edcce45/roles/9223a9bb40484faca3a7bcfeb41a2215` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:06.557696 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a1f08c-2568-4fc4-bd18-6c7448fd469b None admin] RBAC: Authorizing `identity:create_grant(project_id=fce7a22bc8ff438d90035074a2c73590, user_id=0773b663533d44ba8d001f743edcce45, role_id=9223a9bb40484faca3a7bcfeb41a2215)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:06.559595 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a1f08c-2568-4fc4-bd18-6c7448fd469b None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:06.578095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 117/231] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jul 26 04:46:06 2022] PUT /identity/v3/projects/fce7a22bc8ff438d90035074a2c73590/users/0773b663533d44ba8d001f743edcce45/roles/9223a9bb40484faca3a7bcfeb41a2215 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 04:46:06.642506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 115/232] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 04:46:06.649404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1f08c-2568-4fc4-bd18-6c7448fd469b None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:06.774215 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1f08c-2568-4fc4-bd18-6c7448fd469b None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:06.807338 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:06.813317 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966a721-be51-4ee9-90c3-0abedb341d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:06.814350 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966a721-be51-4ee9-90c3-0abedb341d2e None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:06.814932 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966a721-be51-4ee9-90c3-0abedb341d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:06.815400 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966a721-be51-4ee9-90c3-0abedb341d2e None admin] PATH_INFO: `/v3/projects/fce7a22bc8ff438d90035074a2c73590` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:06.817465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3966a721-be51-4ee9-90c3-0abedb341d2e None admin] RBAC: Authorizing `identity:get_project(project_id=fce7a22bc8ff438d90035074a2c73590)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:06.819427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3966a721-be51-4ee9-90c3-0abedb341d2e None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:06.826402 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 118/233] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:06 2022] GET /identity/v3/projects/fce7a22bc8ff438d90035074a2c73590 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:06.827900 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55518e-d356-4b9a-9926-db854a00d275 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:06.887117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:06.937312 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55518e-d356-4b9a-9926-db854a00d275 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:06.974712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb2da22-8210-406f-838c-33de4a700ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:06.975767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2da22-8210-406f-838c-33de4a700ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:06.976343 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2da22-8210-406f-838c-33de4a700ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:06.976821 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2da22-8210-406f-838c-33de4a700ef0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:06.977877 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb2da22-8210-406f-838c-33de4a700ef0 None admin] RBAC: Adding query filter params (role.id=7519356859fb40508c25bec8161207fd, scope.project.id=fce7a22bc8ff438d90035074a2c73590, user.id=a6ab10409e8d4a1fa6350a506c28aa68) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:06.978403 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb2da22-8210-406f-838c-33de4a700ef0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:06.980158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb2da22-8210-406f-838c-33de4a700ef0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:06.986770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 116/234] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:46:06 2022] GET /identity/v3/role_assignments?user.id=a6ab10409e8d4a1fa6350a506c28aa68&scope.project.id=fce7a22bc8ff438d90035074a2c73590&role.id=7519356859fb40508c25bec8161207fd => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:08.164253 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72520908-c547-4bf6-ad06-fb56df9f76a2 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:08.164787 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72520908-c547-4bf6-ad06-fb56df9f76a2 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:08.164787 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72520908-c547-4bf6-ad06-fb56df9f76a2 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:08.166450 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 119/235] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:08.175839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab39a4-eeaa-4c13-a138-01e940b2bd9a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:08.176317 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab39a4-eeaa-4c13-a138-01e940b2bd9a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:08.176390 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab39a4-eeaa-4c13-a138-01e940b2bd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:08.204663 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e0ab39a4-eeaa-4c13-a138-01e940b2bd9a None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:08.205906 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e0ab39a4-eeaa-4c13-a138-01e940b2bd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:08.302578 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 117/236] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:08.357235 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72520908-c547-4bf6-ad06-fb56df9f76a2 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:08.503395 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72520908-c547-4bf6-ad06-fb56df9f76a2 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:08.546706 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d83318-2d72-4261-8118-4de298e6545a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:08.547365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d83318-2d72-4261-8118-4de298e6545a None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:08.547687 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d83318-2d72-4261-8118-4de298e6545a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:08.547963 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d83318-2d72-4261-8118-4de298e6545a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:08.558861 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d83318-2d72-4261-8118-4de298e6545a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:08.564445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-41d83318-2d72-4261-8118-4de298e6545a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:46:08.564445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:08.564445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:08.564445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:08.564445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:08.564445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:08.564445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:08.564445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:08.564445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:08.564445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:08.564445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:08.564445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:08.564445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:08.564445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:08.564445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:08.564445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:08.565360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:08.565360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:08.565360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:08.565360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:08.565360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:08.565360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:08.565360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:08.565360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:08.565360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:08.565360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:08.565360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:08.565360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:08.565360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:08.565360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:08.565360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:08.565360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:08.566173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:08.566173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:08.566173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:08.566173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:08.566173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:08.566173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:08.566173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:08.566173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:08.566173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:08.566173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:08.566173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:46:08.566173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:08.570811 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 120/237] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:46:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:08.572428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab39a4-eeaa-4c13-a138-01e940b2bd9a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:08.673711 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:08.719081 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab39a4-eeaa-4c13-a138-01e940b2bd9a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:08.759093 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f8cdd-ad7f-47b3-bcab-22815bcecf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:08.759676 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f8cdd-ad7f-47b3-bcab-22815bcecf97 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:08.760012 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f8cdd-ad7f-47b3-bcab-22815bcecf97 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:08.760284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f8cdd-ad7f-47b3-bcab-22815bcecf97 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:08.761055 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f8cdd-ad7f-47b3-bcab-22815bcecf97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:08.761364 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f8cdd-ad7f-47b3-bcab-22815bcecf97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:08.762744 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f8cdd-ad7f-47b3-bcab-22815bcecf97 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:08.774922 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 118/238] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:08.777421 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d83318-2d72-4261-8118-4de298e6545a None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:08.886999 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:08.945862 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d83318-2d72-4261-8118-4de298e6545a None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:08.995337 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c374a04-2d2d-492c-876e-17a0ffa9d5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:08.996026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c374a04-2d2d-492c-876e-17a0ffa9d5fc None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:08.996310 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c374a04-2d2d-492c-876e-17a0ffa9d5fc None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:08.996583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c374a04-2d2d-492c-876e-17a0ffa9d5fc None admin] PATH_INFO: `/v3/users/0773b663533d44ba8d001f743edcce45` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:09.000294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c374a04-2d2d-492c-876e-17a0ffa9d5fc None admin] RBAC: Authorizing `identity:get_user(user_id=0773b663533d44ba8d001f743edcce45)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:09.001803 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c374a04-2d2d-492c-876e-17a0ffa9d5fc None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:09.003944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 121/239] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:46:08 2022] GET /identity/v3/users/0773b663533d44ba8d001f743edcce45 => generated 283 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:09.009982 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f8cdd-ad7f-47b3-bcab-22815bcecf97 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:09.066378 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972b11a-121f-4cd9-85f9-97e4a11132db None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:09.066691 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972b11a-121f-4cd9-85f9-97e4a11132db None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:09.066931 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972b11a-121f-4cd9-85f9-97e4a11132db None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:09.068636 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 122/240] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:46:09.077356 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62434425-0000-4ec9-bd11-46af1bde1920 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:09.077644 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62434425-0000-4ec9-bd11-46af1bde1920 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:09.077921 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62434425-0000-4ec9-bd11-46af1bde1920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:09.112670 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-62434425-0000-4ec9-bd11-46af1bde1920 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:09.113908 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-62434425-0000-4ec9-bd11-46af1bde1920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:09.158188 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f8cdd-ad7f-47b3-bcab-22815bcecf97 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:09.176845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:09.189670 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 123/241] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:09.190981 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f7288-4256-41e6-b82a-422e5c855af6 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:09.191271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f7288-4256-41e6-b82a-422e5c855af6 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:09.191632 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f7288-4256-41e6-b82a-422e5c855af6 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:09.192942 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 124/242] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:09.198492 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20975fb5-c207-49a6-8fd0-c67faa142f8e None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:09.198780 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20975fb5-c207-49a6-8fd0-c67faa142f8e None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:09.199059 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20975fb5-c207-49a6-8fd0-c67faa142f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:09.217516 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-20975fb5-c207-49a6-8fd0-c67faa142f8e None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:09.218704 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-20975fb5-c207-49a6-8fd0-c67faa142f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:09.227379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a312b719-b712-4be9-b54b-fc68bb4e76db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:09.228475 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312b719-b712-4be9-b54b-fc68bb4e76db None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:09.228958 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312b719-b712-4be9-b54b-fc68bb4e76db None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:09.229434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312b719-b712-4be9-b54b-fc68bb4e76db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:09.276063 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:09.295670 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 125/243] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 04:46:09.296626 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20975fb5-c207-49a6-8fd0-c67faa142f8e None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:09.350196 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a312b719-b712-4be9-b54b-fc68bb4e76db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:09.352937 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a312b719-b712-4be9-b54b-fc68bb4e76db None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:09.426814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20975fb5-c207-49a6-8fd0-c67faa142f8e None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:09.473991 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141df3f7-d9a1-4091-a137-ccc8385bee3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:09.475073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141df3f7-d9a1-4091-a137-ccc8385bee3f None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:09.475583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141df3f7-d9a1-4091-a137-ccc8385bee3f None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:09.476066 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141df3f7-d9a1-4091-a137-ccc8385bee3f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:09.492935 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141df3f7-d9a1-4091-a137-ccc8385bee3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:09.495392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 119/244] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:46:09.501003 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516187c-4271-437f-86ad-d83534851694 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:09.501303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516187c-4271-437f-86ad-d83534851694 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:09.501584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516187c-4271-437f-86ad-d83534851694 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:09.502685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-141df3f7-d9a1-4091-a137-ccc8385bee3f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 04:46:09.502685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:09.502685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:09.502685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:09.502685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:09.502685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:09.502685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:09.502685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:09.502685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:09.502685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:09.502685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:09.502685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:09.502685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:09.502685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:09.502685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:09.502685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:09.503531 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:09.503531 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:09.503531 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:09.503531 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:09.503531 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:09.503531 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:09.503531 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:09.503531 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:09.503531 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:09.503531 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:09.503531 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:09.503531 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:09.503531 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:09.503531 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:09.503531 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:09.503531 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:09.504729 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:09.504729 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:09.504729 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:09.504729 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:09.504729 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:09.504729 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:09.504729 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:09.504729 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:09.504729 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:09.504729 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:09.504729 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 04:46:09.504729 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:09.505546 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 126/245] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:09.507016 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 120/246] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:09.507851 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516187c-4271-437f-86ad-d83534851694 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:09.508873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141df3f7-d9a1-4091-a137-ccc8385bee3f None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:09.524650 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:09.562790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:09.618046 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141df3f7-d9a1-4091-a137-ccc8385bee3f None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:09.631091 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516187c-4271-437f-86ad-d83534851694 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:09.657759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a571df-2467-4e0e-ad72-54b7f8be6e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:09.658426 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a571df-2467-4e0e-ad72-54b7f8be6e1c None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:09.658710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a571df-2467-4e0e-ad72-54b7f8be6e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:09.658981 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a571df-2467-4e0e-ad72-54b7f8be6e1c None admin] PATH_INFO: `/v3/projects/fce7a22bc8ff438d90035074a2c73590` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:09.660225 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a571df-2467-4e0e-ad72-54b7f8be6e1c None admin] RBAC: Authorizing `identity:get_project(project_id=fce7a22bc8ff438d90035074a2c73590)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:09.661711 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a571df-2467-4e0e-ad72-54b7f8be6e1c None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:09.670510 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 127/247] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:09 2022] GET /identity/v3/projects/fce7a22bc8ff438d90035074a2c73590 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:09.670510 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8afba-7c83-4fa8-bde4-35a1eee965f3 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:09.670510 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8afba-7c83-4fa8-bde4-35a1eee965f3 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:09.670510 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8afba-7c83-4fa8-bde4-35a1eee965f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:09.681584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073752c-8e05-4c7e-9c68-72ff8fb66c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:09.682262 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073752c-8e05-4c7e-9c68-72ff8fb66c73 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:09.682550 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073752c-8e05-4c7e-9c68-72ff8fb66c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:09.682828 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073752c-8e05-4c7e-9c68-72ff8fb66c73 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:09.684160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7073752c-8e05-4c7e-9c68-72ff8fb66c73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:09.685711 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7073752c-8e05-4c7e-9c68-72ff8fb66c73 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:09.689599 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 121/248] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:09.689599 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073752c-8e05-4c7e-9c68-72ff8fb66c73 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:09.694996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-21e8afba-7c83-4fa8-bde4-35a1eee965f3 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:09.696329 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-21e8afba-7c83-4fa8-bde4-35a1eee965f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:09.767649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 128/249] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:09.775817 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8afba-7c83-4fa8-bde4-35a1eee965f3 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:09.775999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:09.798997 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073752c-8e05-4c7e-9c68-72ff8fb66c73 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:09.816062 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:09.833549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35174231-64ed-4a23-bb80-5e5c474982f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:09.834151 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35174231-64ed-4a23-bb80-5e5c474982f7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:09.834424 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35174231-64ed-4a23-bb80-5e5c474982f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:09.834764 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35174231-64ed-4a23-bb80-5e5c474982f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:09.835413 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35174231-64ed-4a23-bb80-5e5c474982f7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:09.835755 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35174231-64ed-4a23-bb80-5e5c474982f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:09.837166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35174231-64ed-4a23-bb80-5e5c474982f7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:09.846111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 122/250] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:46:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:09.847222 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35174231-64ed-4a23-bb80-5e5c474982f7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:09.870196 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8afba-7c83-4fa8-bde4-35a1eee965f3 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:09.906044 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa17dc-ef13-4d8b-b0fd-30acc4c0ee94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:09.906567 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa17dc-ef13-4d8b-b0fd-30acc4c0ee94 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:09.906855 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa17dc-ef13-4d8b-b0fd-30acc4c0ee94 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:09.907121 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa17dc-ef13-4d8b-b0fd-30acc4c0ee94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:09.907978 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa17dc-ef13-4d8b-b0fd-30acc4c0ee94 None admin] RBAC: Adding query filter params (role.id=72bf8bd23d2c4b76ad32b7b35d6db412, scope.project.id=fce7a22bc8ff438d90035074a2c73590, user.id=0773b663533d44ba8d001f743edcce45) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:09.908287 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa17dc-ef13-4d8b-b0fd-30acc4c0ee94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:09.909767 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa17dc-ef13-4d8b-b0fd-30acc4c0ee94 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:09.915710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 129/251] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:46:09 2022] GET /identity/v3/role_assignments?user.id=0773b663533d44ba8d001f743edcce45&scope.project.id=fce7a22bc8ff438d90035074a2c73590&role.id=72bf8bd23d2c4b76ad32b7b35d6db412 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:09.918159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4c3a4-69a0-499d-87c6-858de63e4414 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:09.918452 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4c3a4-69a0-499d-87c6-858de63e4414 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:09.918729 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4c3a4-69a0-499d-87c6-858de63e4414 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:09.920278 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 130/252] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:09.921012 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a4c3a4-69a0-499d-87c6-858de63e4414 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:09.954566 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35174231-64ed-4a23-bb80-5e5c474982f7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:10.006110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e9baf-2efa-429e-bf50-d4d8e17553ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:10.007028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e9baf-2efa-429e-bf50-d4d8e17553ac None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:10.007464 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e9baf-2efa-429e-bf50-d4d8e17553ac None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:10.007923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e9baf-2efa-429e-bf50-d4d8e17553ac None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:10.008818 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290e9baf-2efa-429e-bf50-d4d8e17553ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:10.010576 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290e9baf-2efa-429e-bf50-d4d8e17553ac None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:10.023015 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:10.036934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 123/253] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jul 26 04:46:09 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:46:10.039221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e9baf-2efa-429e-bf50-d4d8e17553ac None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:10.055163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a4c3a4-69a0-499d-87c6-858de63e4414 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:10.072873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:10.112893 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bde500-41ef-4b13-aaee-b3bf0439266d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:10.113902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bde500-41ef-4b13-aaee-b3bf0439266d None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:10.114430 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bde500-41ef-4b13-aaee-b3bf0439266d None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:10.114845 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bde500-41ef-4b13-aaee-b3bf0439266d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:10.128612 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bde500-41ef-4b13-aaee-b3bf0439266d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:10.134181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-e5bde500-41ef-4b13-aaee-b3bf0439266d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 04:46:10.134181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:10.134181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:10.134181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:10.134181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:10.134181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:10.134181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:10.134181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:10.134181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:10.134181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:10.134181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:10.134181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:10.134181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:10.134181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:10.134181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:10.134181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:10.135365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:10.135365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:10.135365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:10.135365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:10.135365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:10.135365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:10.135365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:10.135365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:10.135365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:10.135365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:10.135365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:10.135365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:10.135365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:10.135365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:10.135365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:10.135365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:10.136316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:10.136316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:10.136316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:10.136316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:10.136316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:10.136316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:10.136316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:10.136316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:10.136316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:10.136316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:10.136316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 04:46:10.136316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:10.138410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 131/254] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jul 26 04:46:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:10.140259 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cbcc2-3d22-48ce-ab8e-3abe5242354e None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:10.140713 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cbcc2-3d22-48ce-ab8e-3abe5242354e None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:10.141155 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cbcc2-3d22-48ce-ab8e-3abe5242354e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:10.165822 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-773cbcc2-3d22-48ce-ab8e-3abe5242354e None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:10.165822 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-773cbcc2-3d22-48ce-ab8e-3abe5242354e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:10.197655 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e9baf-2efa-429e-bf50-d4d8e17553ac None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:10.241636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59340be2-4e4d-48fa-acb7-1f5288be11f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:10.242308 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59340be2-4e4d-48fa-acb7-1f5288be11f8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:10.242593 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59340be2-4e4d-48fa-acb7-1f5288be11f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:10.242866 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59340be2-4e4d-48fa-acb7-1f5288be11f8 None admin] PATH_INFO: `/v3/users/e1d7022163484a0082d4869a9b9d6b6d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:10.245636 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 132/255] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:10.245636 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cbcc2-3d22-48ce-ab8e-3abe5242354e None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:10.270028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59340be2-4e4d-48fa-acb7-1f5288be11f8 None admin] RBAC: Authorizing `identity:get_user(user_id=e1d7022163484a0082d4869a9b9d6b6d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:10.271542 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59340be2-4e4d-48fa-acb7-1f5288be11f8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:10.275522 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 124/256] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:46:10 2022] GET /identity/v3/users/e1d7022163484a0082d4869a9b9d6b6d => generated 297 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:10.276394 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59340be2-4e4d-48fa-acb7-1f5288be11f8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:10.350887 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:10.384354 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:10.390215 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cbcc2-3d22-48ce-ab8e-3abe5242354e None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:10.417229 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59340be2-4e4d-48fa-acb7-1f5288be11f8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:10.440377 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f13bc-4e4e-40b4-b3c8-915944d2c281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:10.441026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f13bc-4e4e-40b4-b3c8-915944d2c281 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:10.441310 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f13bc-4e4e-40b4-b3c8-915944d2c281 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:10.441585 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f13bc-4e4e-40b4-b3c8-915944d2c281 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:10.442619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f13bc-4e4e-40b4-b3c8-915944d2c281 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:10.442928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f13bc-4e4e-40b4-b3c8-915944d2c281 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:10.444954 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f13bc-4e4e-40b4-b3c8-915944d2c281 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:10.455591 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 133/257] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:46:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:10.456411 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f13bc-4e4e-40b4-b3c8-915944d2c281 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:10.459987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca30879-af99-42f4-a668-dbfe526a5390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:10.461017 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca30879-af99-42f4-a668-dbfe526a5390 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:10.461298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca30879-af99-42f4-a668-dbfe526a5390 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:10.461573 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca30879-af99-42f4-a668-dbfe526a5390 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:10.475205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca30879-af99-42f4-a668-dbfe526a5390 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:10.480232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-0ca30879-af99-42f4-a668-dbfe526a5390 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:46:10.480232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:10.480232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:10.480232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:10.480232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:10.480232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:10.480232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:10.480232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:10.480232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:10.480232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:10.480232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:10.480232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:10.480232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:10.480232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:10.480232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:10.480232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:10.481340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:10.481340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:10.481340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:10.481340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:10.481340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:10.481340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:10.481340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:10.481340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:10.481340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:10.481340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:10.481340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:10.481340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:10.481340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:10.481340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:10.481340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:10.481340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:10.482119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:10.482119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:10.482119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:10.482119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:10.482119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:10.482119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:10.482119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:10.482119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:10.482119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:10.482119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:10.482119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:46:10.482119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:10.482119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 125/258] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:10.482800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca30879-af99-42f4-a668-dbfe526a5390 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:10.582075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f13bc-4e4e-40b4-b3c8-915944d2c281 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:10.600674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca30879-af99-42f4-a668-dbfe526a5390 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:10.616842 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:10.633009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929a669-e657-4656-b9df-3f0f23901cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:10.633954 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929a669-e657-4656-b9df-3f0f23901cfc None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:10.634238 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929a669-e657-4656-b9df-3f0f23901cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:10.634557 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929a669-e657-4656-b9df-3f0f23901cfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:10.651351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474dd278-537c-4d31-87c7-ddd63bdf5004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:10.652131 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dd278-537c-4d31-87c7-ddd63bdf5004 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:10.652417 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dd278-537c-4d31-87c7-ddd63bdf5004 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:10.652694 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dd278-537c-4d31-87c7-ddd63bdf5004 None admin] PATH_INFO: `/v3/users/0773b663533d44ba8d001f743edcce45` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:10.654358 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474dd278-537c-4d31-87c7-ddd63bdf5004 None admin] RBAC: Authorizing `identity:get_user(user_id=0773b663533d44ba8d001f743edcce45)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:10.656200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474dd278-537c-4d31-87c7-ddd63bdf5004 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:10.658669 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 126/259] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:46:10 2022] GET /identity/v3/users/0773b663533d44ba8d001f743edcce45 => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:10.659932 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474dd278-537c-4d31-87c7-ddd63bdf5004 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:10.676416 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:10.727255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b929a669-e657-4656-b9df-3f0f23901cfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:10.729421 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b929a669-e657-4656-b9df-3f0f23901cfc None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:10.793114 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474dd278-537c-4d31-87c7-ddd63bdf5004 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:10.830779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68a320-8765-427b-b731-3c2d6ae828a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:10.831864 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68a320-8765-427b-b731-3c2d6ae828a1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:10.832343 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68a320-8765-427b-b731-3c2d6ae828a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:10.832863 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68a320-8765-427b-b731-3c2d6ae828a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:10.833802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a68a320-8765-427b-b731-3c2d6ae828a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:10.834328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a68a320-8765-427b-b731-3c2d6ae828a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:10.835984 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a68a320-8765-427b-b731-3c2d6ae828a1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:10.851314 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 127/260] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:46:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:10.852616 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68a320-8765-427b-b731-3c2d6ae828a1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:10.853919 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 134/261] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:46:10.854695 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929a669-e657-4656-b9df-3f0f23901cfc None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:10.904144 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:10.940733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:10.981066 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68a320-8765-427b-b731-3c2d6ae828a1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:10.993270 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929a669-e657-4656-b9df-3f0f23901cfc None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:11.028470 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170bd984-9233-441c-a886-358f63466aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:11.029636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bd984-9233-441c-a886-358f63466aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:11.030186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bd984-9233-441c-a886-358f63466aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:11.030822 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bd984-9233-441c-a886-358f63466aa7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:11.032198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3192da-d28a-490a-9162-2f2ca66bb7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:11.032872 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3192da-d28a-490a-9162-2f2ca66bb7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:11.033158 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3192da-d28a-490a-9162-2f2ca66bb7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:11.033441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3192da-d28a-490a-9162-2f2ca66bb7b1 None admin] PATH_INFO: `/v3/users/a6ab10409e8d4a1fa6350a506c28aa68` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:11.034708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3192da-d28a-490a-9162-2f2ca66bb7b1 None admin] RBAC: Authorizing `identity:get_user(user_id=a6ab10409e8d4a1fa6350a506c28aa68)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:11.036935 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3192da-d28a-490a-9162-2f2ca66bb7b1 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:11.042363 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 135/262] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:46:10 2022] GET /identity/v3/users/a6ab10409e8d4a1fa6350a506c28aa68 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:11.042363 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3192da-d28a-490a-9162-2f2ca66bb7b1 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:11.127398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170bd984-9233-441c-a886-358f63466aa7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:11.129840 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170bd984-9233-441c-a886-358f63466aa7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:11.154631 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3192da-d28a-490a-9162-2f2ca66bb7b1 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:11.182572 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:11.203714 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:11.211645 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3610dbc-f86c-4e3c-84bb-bf4cff357584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:11.212312 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3610dbc-f86c-4e3c-84bb-bf4cff357584 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:11.212693 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3610dbc-f86c-4e3c-84bb-bf4cff357584 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:11.212970 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3610dbc-f86c-4e3c-84bb-bf4cff357584 None admin] PATH_INFO: `/v3/projects/fce7a22bc8ff438d90035074a2c73590` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:11.214312 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3610dbc-f86c-4e3c-84bb-bf4cff357584 None admin] RBAC: Authorizing `identity:get_project(project_id=fce7a22bc8ff438d90035074a2c73590)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:11.215835 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3610dbc-f86c-4e3c-84bb-bf4cff357584 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:11.218112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 136/263] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:11 2022] GET /identity/v3/projects/fce7a22bc8ff438d90035074a2c73590 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:11.218810 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3610dbc-f86c-4e3c-84bb-bf4cff357584 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:11.270772 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 128/264] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:46:11.271702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170bd984-9233-441c-a886-358f63466aa7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:11.351189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3610dbc-f86c-4e3c-84bb-bf4cff357584 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:11.401848 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170bd984-9233-441c-a886-358f63466aa7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:11.407154 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee778390-7541-43cf-810e-bc3e44c9fd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:11.407779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee778390-7541-43cf-810e-bc3e44c9fd92 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:11.408066 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee778390-7541-43cf-810e-bc3e44c9fd92 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:11.408342 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee778390-7541-43cf-810e-bc3e44c9fd92 None admin] PATH_INFO: `/v3/projects/fce7a22bc8ff438d90035074a2c73590` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:11.409726 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee778390-7541-43cf-810e-bc3e44c9fd92 None admin] RBAC: Authorizing `identity:get_project(project_id=fce7a22bc8ff438d90035074a2c73590)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:11.411254 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee778390-7541-43cf-810e-bc3e44c9fd92 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:11.413886 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 137/265] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:11 2022] GET /identity/v3/projects/fce7a22bc8ff438d90035074a2c73590 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:11.414199 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee778390-7541-43cf-810e-bc3e44c9fd92 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:11.447014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67e820-c15c-457d-a1fe-7589fcb75d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:11.448479 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67e820-c15c-457d-a1fe-7589fcb75d62 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:11.449008 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67e820-c15c-457d-a1fe-7589fcb75d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:11.449716 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67e820-c15c-457d-a1fe-7589fcb75d62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:11.451491 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd67e820-c15c-457d-a1fe-7589fcb75d62 None admin] RBAC: Adding query filter params (role.id=6c31395d654e4add9c982f62b79c2873, scope.project.id=fce7a22bc8ff438d90035074a2c73590, user.id=e1d7022163484a0082d4869a9b9d6b6d) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:11.452292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd67e820-c15c-457d-a1fe-7589fcb75d62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:11.454744 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd67e820-c15c-457d-a1fe-7589fcb75d62 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:11.464189 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 129/266] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:46:11 2022] GET /identity/v3/role_assignments?user.id=e1d7022163484a0082d4869a9b9d6b6d&scope.project.id=fce7a22bc8ff438d90035074a2c73590&role.id=6c31395d654e4add9c982f62b79c2873 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:11.465501 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67e820-c15c-457d-a1fe-7589fcb75d62 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:11.478219 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:11.505668 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:11.549848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee778390-7541-43cf-810e-bc3e44c9fd92 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:11.602026 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67e820-c15c-457d-a1fe-7589fcb75d62 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:11.618830 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394c65b-0eb6-41f3-b0c5-8c94b63cae70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:11.620025 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394c65b-0eb6-41f3-b0c5-8c94b63cae70 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:11.620527 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394c65b-0eb6-41f3-b0c5-8c94b63cae70 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:11.620993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394c65b-0eb6-41f3-b0c5-8c94b63cae70 None admin] PATH_INFO: `/v3/projects/fce7a22bc8ff438d90035074a2c73590` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:11.628983 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394c65b-0eb6-41f3-b0c5-8c94b63cae70 None admin] RBAC: Authorizing `identity:get_project(project_id=fce7a22bc8ff438d90035074a2c73590)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:11.631084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394c65b-0eb6-41f3-b0c5-8c94b63cae70 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:11.633341 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 138/267] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:11 2022] GET /identity/v3/projects/fce7a22bc8ff438d90035074a2c73590 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:11.641853 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394c65b-0eb6-41f3-b0c5-8c94b63cae70 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:11.647743 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad49de9-d43e-4013-927e-c8892a1bc908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:11.648471 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad49de9-d43e-4013-927e-c8892a1bc908 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:11.648760 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad49de9-d43e-4013-927e-c8892a1bc908 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:11.649044 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad49de9-d43e-4013-927e-c8892a1bc908 None admin] PATH_INFO: `/v3/projects/fce7a22bc8ff438d90035074a2c73590/users/a6ab10409e8d4a1fa6350a506c28aa68/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:11.652624 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad49de9-d43e-4013-927e-c8892a1bc908 None admin] RBAC: Authorizing `identity:create_grant(project_id=fce7a22bc8ff438d90035074a2c73590, user_id=a6ab10409e8d4a1fa6350a506c28aa68, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:11.654127 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad49de9-d43e-4013-927e-c8892a1bc908 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:11.676013 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 130/268] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jul 26 04:46:11 2022] PUT /identity/v3/projects/fce7a22bc8ff438d90035074a2c73590/users/a6ab10409e8d4a1fa6350a506c28aa68/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:46:11.771691 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394c65b-0eb6-41f3-b0c5-8c94b63cae70 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:11.811042 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e964a-bedd-444f-b73c-b92980be8123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:11.811788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e964a-bedd-444f-b73c-b92980be8123 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:11.812077 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e964a-bedd-444f-b73c-b92980be8123 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:11.812358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e964a-bedd-444f-b73c-b92980be8123 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:11.813206 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e964a-bedd-444f-b73c-b92980be8123 None admin] RBAC: Adding query filter params (role.id=9223a9bb40484faca3a7bcfeb41a2215, scope.project.id=fce7a22bc8ff438d90035074a2c73590, user.id=0773b663533d44ba8d001f743edcce45) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:11.813523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e964a-bedd-444f-b73c-b92980be8123 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:11.815041 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e964a-bedd-444f-b73c-b92980be8123 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:11.817302 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:11.827597 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 139/269] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:46:11 2022] GET /identity/v3/role_assignments?user.id=0773b663533d44ba8d001f743edcce45&scope.project.id=fce7a22bc8ff438d90035074a2c73590&role.id=9223a9bb40484faca3a7bcfeb41a2215 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:12.402508 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f78f8-5ea7-4b2e-b0c0-439dbe557766 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:12.403019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f78f8-5ea7-4b2e-b0c0-439dbe557766 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:12.403090 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f78f8-5ea7-4b2e-b0c0-439dbe557766 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:12.404801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 131/270] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:12.410497 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a1b10-57a3-4e39-bde5-37cf2d973f71 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:12.411384 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a1b10-57a3-4e39-bde5-37cf2d973f71 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:12.411898 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a1b10-57a3-4e39-bde5-37cf2d973f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:12.431193 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-7c2a1b10-57a3-4e39-bde5-37cf2d973f71 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:12.432757 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-7c2a1b10-57a3-4e39-bde5-37cf2d973f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:12.439586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0866f2-e1b3-4564-9835-9a608cbc11ae None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:12.439586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0866f2-e1b3-4564-9835-9a608cbc11ae None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:12.439845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0866f2-e1b3-4564-9835-9a608cbc11ae None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:12.441281 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 132/271] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:12.447159 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028446ee-857d-462d-be45-85631abfd449 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:12.447741 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028446ee-857d-462d-be45-85631abfd449 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:12.448237 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028446ee-857d-462d-be45-85631abfd449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:12.479003 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-028446ee-857d-462d-be45-85631abfd449 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:12.481424 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-028446ee-857d-462d-be45-85631abfd449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:12.519142 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 140/272] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 04:46:12.542045 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a1b10-57a3-4e39-bde5-37cf2d973f71 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:12.561153 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:12.575939 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 133/273] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 04:46:12.601493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028446ee-857d-462d-be45-85631abfd449 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:12.661091 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a1b10-57a3-4e39-bde5-37cf2d973f71 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:12.689367 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:12.698636 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ec696-4d27-47c4-9117-45912a7b9068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:12.699353 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ec696-4d27-47c4-9117-45912a7b9068 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:12.699659 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ec696-4d27-47c4-9117-45912a7b9068 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:12.699934 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ec696-4d27-47c4-9117-45912a7b9068 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:12.712147 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ec696-4d27-47c4-9117-45912a7b9068 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:12.718944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-b96ec696-4d27-47c4-9117-45912a7b9068 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:46:12.718944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:12.718944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:12.718944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:12.718944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:12.718944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:12.718944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:12.718944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:12.718944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:12.718944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:12.718944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:12.718944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:12.718944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:12.718944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:12.718944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:12.718944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:12.720242 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:12.720242 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:12.720242 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:12.720242 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:12.720242 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:12.720242 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:12.720242 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:12.720242 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:12.720242 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:12.720242 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:12.720242 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:12.720242 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:12.720242 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:12.720242 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:12.720242 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:12.720242 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:12.721026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:12.721026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:12.721026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:12.721026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:12.721026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:12.721026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:12.721026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:12.721026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:12.721026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:12.721026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:12.721026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:46:12.721026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:12.723644 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 141/274] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:46:12.727522 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ec696-4d27-47c4-9117-45912a7b9068 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:12.736767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028446ee-857d-462d-be45-85631abfd449 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:12.773506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d80b64-ca49-4432-b39d-ceb5d00ab049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:12.774155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d80b64-ca49-4432-b39d-ceb5d00ab049 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:12.774433 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d80b64-ca49-4432-b39d-ceb5d00ab049 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:12.774710 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d80b64-ca49-4432-b39d-ceb5d00ab049 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:12.776031 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d80b64-ca49-4432-b39d-ceb5d00ab049 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:12.777640 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d80b64-ca49-4432-b39d-ceb5d00ab049 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:12.779446 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 134/275] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:12.785266 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d80b64-ca49-4432-b39d-ceb5d00ab049 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:12.853566 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:12.873709 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ec696-4d27-47c4-9117-45912a7b9068 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:12.896503 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d80b64-ca49-4432-b39d-ceb5d00ab049 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:12.920640 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857dca9-11e4-40e1-b646-6112823bcf87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:12.921826 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857dca9-11e4-40e1-b646-6112823bcf87 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:12.922740 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857dca9-11e4-40e1-b646-6112823bcf87 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:12.923219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857dca9-11e4-40e1-b646-6112823bcf87 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:12.924492 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857dca9-11e4-40e1-b646-6112823bcf87 None admin] RBAC: Adding query filter params (name=member) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:12.925019 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857dca9-11e4-40e1-b646-6112823bcf87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:12.927376 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857dca9-11e4-40e1-b646-6112823bcf87 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:12.935133 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30734cf-5c6c-42b0-8c7e-693c7b9317e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:12.935867 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30734cf-5c6c-42b0-8c7e-693c7b9317e9 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:12.936164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30734cf-5c6c-42b0-8c7e-693c7b9317e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:12.936458 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30734cf-5c6c-42b0-8c7e-693c7b9317e9 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:12.937127 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30734cf-5c6c-42b0-8c7e-693c7b9317e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:12.938589 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30734cf-5c6c-42b0-8c7e-693c7b9317e9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:12.939577 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 142/276] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:46:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:12.944837 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857dca9-11e4-40e1-b646-6112823bcf87 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:12.971985 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 135/277] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jul 26 04:46:12 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:46:13.009656 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:13.069796 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857dca9-11e4-40e1-b646-6112823bcf87 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:13.113445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f56a1-601c-4444-b975-8b0b752c1869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:13.114494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f56a1-601c-4444-b975-8b0b752c1869 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:13.114975 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f56a1-601c-4444-b975-8b0b752c1869 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:13.115470 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f56a1-601c-4444-b975-8b0b752c1869 None admin] PATH_INFO: `/v3/users/2f37b195dfb547c58a73c2bfed97b610` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:13.132060 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f56a1-601c-4444-b975-8b0b752c1869 None admin] RBAC: Authorizing `identity:get_user(user_id=2f37b195dfb547c58a73c2bfed97b610)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:13.134066 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f56a1-601c-4444-b975-8b0b752c1869 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:13.136559 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 143/278] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:46:12 2022] GET /identity/v3/users/2f37b195dfb547c58a73c2bfed97b610 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:13.145379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30734cf-5c6c-42b0-8c7e-693c7b9317e9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:13.263066 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30734cf-5c6c-42b0-8c7e-693c7b9317e9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:13.284973 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:13.303943 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928f0bd-2bef-44cc-b23b-e9c8d413aeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:13.304794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928f0bd-2bef-44cc-b23b-e9c8d413aeb1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:13.305084 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928f0bd-2bef-44cc-b23b-e9c8d413aeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:13.305466 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928f0bd-2bef-44cc-b23b-e9c8d413aeb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:13.306621 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0928f0bd-2bef-44cc-b23b-e9c8d413aeb1 None admin] RBAC: Adding query filter params (role.id=7519356859fb40508c25bec8161207fd, user.id=2f37b195dfb547c58a73c2bfed97b610, scope.system=all) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:13.306950 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0928f0bd-2bef-44cc-b23b-e9c8d413aeb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:13.308684 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0928f0bd-2bef-44cc-b23b-e9c8d413aeb1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:13.315781 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 136/279] 10.222.0.17 () {56 vars in 1308 bytes} [Tue Jul 26 04:46:13 2022] GET /identity/v3/role_assignments?user.id=2f37b195dfb547c58a73c2bfed97b610&scope.system=all&role.id=7519356859fb40508c25bec8161207fd => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:13.689495 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c51f8-25b3-46ef-8e56-2d08ecb6a061 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:13.690026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c51f8-25b3-46ef-8e56-2d08ecb6a061 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:13.690026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c51f8-25b3-46ef-8e56-2d08ecb6a061 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:13.691774 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 144/280] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:13.697884 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f6eb0-2497-47a2-83a2-d2af4c8ab0e0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:13.698419 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f6eb0-2497-47a2-83a2-d2af4c8ab0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:13.698419 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f6eb0-2497-47a2-83a2-d2af4c8ab0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:13.717732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-098f6eb0-2497-47a2-83a2-d2af4c8ab0e0 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:13.718946 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-098f6eb0-2497-47a2-83a2-d2af4c8ab0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:13.818203 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 137/281] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:13.840520 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c51f8-25b3-46ef-8e56-2d08ecb6a061 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:13.988900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c51f8-25b3-46ef-8e56-2d08ecb6a061 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:14.009889 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:14.044661 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba488b-60f4-41fa-9e1c-75ac21ff2026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:14.045364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba488b-60f4-41fa-9e1c-75ac21ff2026 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:14.045643 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba488b-60f4-41fa-9e1c-75ac21ff2026 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:14.045913 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba488b-60f4-41fa-9e1c-75ac21ff2026 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:14.068040 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ba488b-60f4-41fa-9e1c-75ac21ff2026 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:14.073130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-29ba488b-60f4-41fa-9e1c-75ac21ff2026 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 04:46:14.073130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:14.073130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:14.073130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:14.073130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:14.073130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:14.073130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:14.073130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:14.073130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:14.073130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:14.073130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:14.073130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:14.073130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:14.073130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:14.073130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:14.073130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:14.074062 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:14.074062 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:14.074062 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:14.074062 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:14.074062 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:14.074062 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:14.074062 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:14.074062 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:14.074062 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:14.074062 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:14.074062 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:14.074062 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:14.074062 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:14.074062 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:14.074062 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:14.074062 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:14.074851 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:14.074851 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:14.074851 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:14.074851 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:14.074851 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:14.074851 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:14.074851 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:14.074851 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:14.074851 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:14.074851 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:14.074851 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 04:46:14.074851 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:14.074851 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 145/282] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:14.080719 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f6eb0-2497-47a2-83a2-d2af4c8ab0e0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:14.122066 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e536934-f890-4507-b9b1-a3cfe3523d6e None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:14.122486 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e536934-f890-4507-b9b1-a3cfe3523d6e None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:14.122768 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e536934-f890-4507-b9b1-a3cfe3523d6e None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:14.124577 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 146/283] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:14.130698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d12554-da6b-4135-af10-d4d467698e7d None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:14.131495 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d12554-da6b-4135-af10-d4d467698e7d None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:14.132798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d12554-da6b-4135-af10-d4d467698e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:14.172801 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-11d12554-da6b-4135-af10-d4d467698e7d None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:14.174065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-11d12554-da6b-4135-af10-d4d467698e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:14.262266 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f6eb0-2497-47a2-83a2-d2af4c8ab0e0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:14.279186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:14.310345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0033afb-e399-45fc-855b-fb04e525709d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:14.311458 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0033afb-e399-45fc-855b-fb04e525709d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:14.312229 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0033afb-e399-45fc-855b-fb04e525709d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:14.313554 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0033afb-e399-45fc-855b-fb04e525709d None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:14.314756 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0033afb-e399-45fc-855b-fb04e525709d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:14.316868 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0033afb-e399-45fc-855b-fb04e525709d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:14.318578 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0033afb-e399-45fc-855b-fb04e525709d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:14.338739 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 138/284] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:46:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:14.345573 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0033afb-e399-45fc-855b-fb04e525709d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:14.368576 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 147/285] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 04:46:14.412777 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d12554-da6b-4135-af10-d4d467698e7d None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:14.523948 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:14.529651 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d12554-da6b-4135-af10-d4d467698e7d None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:14.543067 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0033afb-e399-45fc-855b-fb04e525709d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:14.568188 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69de3b-e5f1-47bd-8b9a-c3d03e0f4a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:14.569094 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69de3b-e5f1-47bd-8b9a-c3d03e0f4a80 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:14.569374 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69de3b-e5f1-47bd-8b9a-c3d03e0f4a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:14.569650 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69de3b-e5f1-47bd-8b9a-c3d03e0f4a80 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:14.582002 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab69de3b-e5f1-47bd-8b9a-c3d03e0f4a80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:14.588165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-ab69de3b-e5f1-47bd-8b9a-c3d03e0f4a80 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:46:14.588165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:14.588165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:14.588165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:14.588165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:14.588165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:14.588165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:14.588165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:14.588165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:14.588165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:14.588165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:14.588165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:14.588165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:14.588165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:14.588165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:14.588165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:14.589074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:14.589074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:14.589074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:14.589074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:14.589074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:14.589074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:14.589074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:14.589074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:14.589074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:14.589074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:14.589074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:14.589074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:14.589074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:14.589074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:14.589074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:14.589074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:14.589846 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:14.589846 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:14.589846 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:14.589846 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:14.589846 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:14.589846 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:14.589846 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:14.589846 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:14.589846 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:14.589846 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:14.589846 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:46:14.589846 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:14.590433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 148/286] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:14.595923 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69de3b-e5f1-47bd-8b9a-c3d03e0f4a80 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:14.618732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a91b88-1c80-4a33-a17e-be6e261bca77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:14.619868 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a91b88-1c80-4a33-a17e-be6e261bca77 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:14.620352 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a91b88-1c80-4a33-a17e-be6e261bca77 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:14.620826 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a91b88-1c80-4a33-a17e-be6e261bca77 None admin] PATH_INFO: `/v3/users/e1d7022163484a0082d4869a9b9d6b6d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:14.622274 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a91b88-1c80-4a33-a17e-be6e261bca77 None admin] RBAC: Authorizing `identity:get_user(user_id=e1d7022163484a0082d4869a9b9d6b6d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:14.624020 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a91b88-1c80-4a33-a17e-be6e261bca77 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:14.625918 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 139/287] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:46:14 2022] GET /identity/v3/users/e1d7022163484a0082d4869a9b9d6b6d => generated 297 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:14.631595 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a91b88-1c80-4a33-a17e-be6e261bca77 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:14.733196 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:14.780427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69de3b-e5f1-47bd-8b9a-c3d03e0f4a80 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:14.799957 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a91b88-1c80-4a33-a17e-be6e261bca77 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:14.836799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b93716-0c7a-4230-b71d-e90c57574861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:14.837830 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b93716-0c7a-4230-b71d-e90c57574861 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:14.838307 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b93716-0c7a-4230-b71d-e90c57574861 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:14.838774 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b93716-0c7a-4230-b71d-e90c57574861 None admin] PATH_INFO: `/v3/projects/fce7a22bc8ff438d90035074a2c73590` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:14.840524 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b93716-0c7a-4230-b71d-e90c57574861 None admin] RBAC: Authorizing `identity:get_project(project_id=fce7a22bc8ff438d90035074a2c73590)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:14.842288 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b93716-0c7a-4230-b71d-e90c57574861 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:14.845011 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 140/288] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:14 2022] GET /identity/v3/projects/fce7a22bc8ff438d90035074a2c73590 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:14.846380 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa7339-99bf-4e59-b066-a2556f0d48bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:14.847200 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa7339-99bf-4e59-b066-a2556f0d48bd None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:14.847662 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa7339-99bf-4e59-b066-a2556f0d48bd None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:14.848064 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa7339-99bf-4e59-b066-a2556f0d48bd None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:14.848929 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa7339-99bf-4e59-b066-a2556f0d48bd None admin] RBAC: Adding query filter params (name=member) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:14.849382 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa7339-99bf-4e59-b066-a2556f0d48bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:14.850902 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b93716-0c7a-4230-b71d-e90c57574861 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:14.851026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa7339-99bf-4e59-b066-a2556f0d48bd None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:14.873040 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 149/289] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:46:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:14.878559 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa7339-99bf-4e59-b066-a2556f0d48bd None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:14.923967 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:14.963076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b93716-0c7a-4230-b71d-e90c57574861 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:14.977115 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:15.000068 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa7339-99bf-4e59-b066-a2556f0d48bd None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:15.039211 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27401e71-bbe2-4f60-96aa-00685a2d4314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:15.040144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27401e71-bbe2-4f60-96aa-00685a2d4314 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:15.040535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b103b2-5ba0-47f5-a631-a6ee8bf680ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:15.040835 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27401e71-bbe2-4f60-96aa-00685a2d4314 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:15.041283 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27401e71-bbe2-4f60-96aa-00685a2d4314 None admin] PATH_INFO: `/v3/users/a6ab10409e8d4a1fa6350a506c28aa68` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:15.041898 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b103b2-5ba0-47f5-a631-a6ee8bf680ec None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:15.042359 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b103b2-5ba0-47f5-a631-a6ee8bf680ec None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:15.042713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b103b2-5ba0-47f5-a631-a6ee8bf680ec None admin] PATH_INFO: `/v3/projects/fce7a22bc8ff438d90035074a2c73590/users/e1d7022163484a0082d4869a9b9d6b6d/roles/6c31395d654e4add9c982f62b79c2873` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:15.043676 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27401e71-bbe2-4f60-96aa-00685a2d4314 None admin] RBAC: Authorizing `identity:get_user(user_id=a6ab10409e8d4a1fa6350a506c28aa68)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:15.045401 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27401e71-bbe2-4f60-96aa-00685a2d4314 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:15.047531 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 150/290] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:46:14 2022] GET /identity/v3/users/a6ab10409e8d4a1fa6350a506c28aa68 => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:15.049114 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b103b2-5ba0-47f5-a631-a6ee8bf680ec None admin] RBAC: Authorizing `identity:create_grant(project_id=fce7a22bc8ff438d90035074a2c73590, user_id=e1d7022163484a0082d4869a9b9d6b6d, role_id=6c31395d654e4add9c982f62b79c2873)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:15.050650 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b103b2-5ba0-47f5-a631-a6ee8bf680ec None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:15.055593 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27401e71-bbe2-4f60-96aa-00685a2d4314 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:15.075800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 141/291] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jul 26 04:46:14 2022] PUT /identity/v3/projects/fce7a22bc8ff438d90035074a2c73590/users/e1d7022163484a0082d4869a9b9d6b6d/roles/6c31395d654e4add9c982f62b79c2873 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:46:15.188736 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:15.204691 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27401e71-bbe2-4f60-96aa-00685a2d4314 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:15.262148 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23250c2-dae4-418d-920a-f1374180f0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:15.262803 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23250c2-dae4-418d-920a-f1374180f0c0 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:15.263088 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23250c2-dae4-418d-920a-f1374180f0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:15.263363 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23250c2-dae4-418d-920a-f1374180f0c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:15.351087 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23250c2-dae4-418d-920a-f1374180f0c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:15.352618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23250c2-dae4-418d-920a-f1374180f0c0 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:15.449670 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:15.465176 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 151/292] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:46:15.471329 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b103b2-5ba0-47f5-a631-a6ee8bf680ec None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:15.621746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b103b2-5ba0-47f5-a631-a6ee8bf680ec None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:15.674912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2a116-705c-4a74-a131-83b26eedbf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:15.676029 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2a116-705c-4a74-a131-83b26eedbf8e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:15.676508 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2a116-705c-4a74-a131-83b26eedbf8e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:15.676991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2a116-705c-4a74-a131-83b26eedbf8e None admin] PATH_INFO: `/v3/projects/fce7a22bc8ff438d90035074a2c73590` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:15.678911 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2a116-705c-4a74-a131-83b26eedbf8e None admin] RBAC: Authorizing `identity:get_project(project_id=fce7a22bc8ff438d90035074a2c73590)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:15.680655 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2a116-705c-4a74-a131-83b26eedbf8e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:15.682655 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 142/293] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:15 2022] GET /identity/v3/projects/fce7a22bc8ff438d90035074a2c73590 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:15.689094 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23250c2-dae4-418d-920a-f1374180f0c0 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:15.787935 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db5a28-9fe8-459e-bbbf-75555ac856d6 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:15.788475 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db5a28-9fe8-459e-bbbf-75555ac856d6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:15.788475 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db5a28-9fe8-459e-bbbf-75555ac856d6 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:15.791092 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 143/294] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:46:15.795345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54c99a-23c1-4cc0-8f61-d36ac46cb626 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:15.795686 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54c99a-23c1-4cc0-8f61-d36ac46cb626 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:15.795967 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54c99a-23c1-4cc0-8f61-d36ac46cb626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:15.798480 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23250c2-dae4-418d-920a-f1374180f0c0 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:15.834449 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11bab2c-568a-4d46-be5f-f4ca80514e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:15.835077 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11bab2c-568a-4d46-be5f-f4ca80514e27 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:15.835181 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6e54c99a-23c1-4cc0-8f61-d36ac46cb626 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:15.835362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11bab2c-568a-4d46-be5f-f4ca80514e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:15.835702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11bab2c-568a-4d46-be5f-f4ca80514e27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:15.836434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6e54c99a-23c1-4cc0-8f61-d36ac46cb626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:15.836607 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11bab2c-568a-4d46-be5f-f4ca80514e27 None admin] RBAC: Adding query filter params (role.id=7519356859fb40508c25bec8161207fd, scope.project.id=fce7a22bc8ff438d90035074a2c73590, user.id=a6ab10409e8d4a1fa6350a506c28aa68) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:15.837079 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11bab2c-568a-4d46-be5f-f4ca80514e27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:15.838621 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11bab2c-568a-4d46-be5f-f4ca80514e27 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:15.846080 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 152/295] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:46:15 2022] GET /identity/v3/role_assignments?user.id=a6ab10409e8d4a1fa6350a506c28aa68&scope.project.id=fce7a22bc8ff438d90035074a2c73590&role.id=7519356859fb40508c25bec8161207fd => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:15.853164 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600dc97c-d02d-4b94-b236-b0e0e11f6a68 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:15.853516 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600dc97c-d02d-4b94-b236-b0e0e11f6a68 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:15.853801 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600dc97c-d02d-4b94-b236-b0e0e11f6a68 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:15.856563 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 153/296] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:15.861299 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870fca1-11a0-4bd6-b483-e3fa93280172 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:15.861581 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870fca1-11a0-4bd6-b483-e3fa93280172 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:15.861851 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870fca1-11a0-4bd6-b483-e3fa93280172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:15.890202 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-e870fca1-11a0-4bd6-b483-e3fa93280172 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:15.891331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-e870fca1-11a0-4bd6-b483-e3fa93280172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:15.924916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:15.967748 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 144/297] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 04:46:15.977108 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf2ab9-76c3-49f1-9005-e686a6f323b9 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:15.977635 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:15.978028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf2ab9-76c3-49f1-9005-e686a6f323b9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:15.978391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf2ab9-76c3-49f1-9005-e686a6f323b9 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:15.980075 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 145/298] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:15 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:15.986519 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786a1e9-229a-438e-85e6-bc4ba1933322 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:15.987303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786a1e9-229a-438e-85e6-bc4ba1933322 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:15.987597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786a1e9-229a-438e-85e6-bc4ba1933322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:15.999501 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 154/299] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 04:46:16.011169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870fca1-11a0-4bd6-b483-e3fa93280172 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:16.025709 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d786a1e9-229a-438e-85e6-bc4ba1933322 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:16.026919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d786a1e9-229a-438e-85e6-bc4ba1933322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:16.131206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 146/300] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 04:46:16.132138 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786a1e9-229a-438e-85e6-bc4ba1933322 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:16.141543 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870fca1-11a0-4bd6-b483-e3fa93280172 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:16.193753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a4fe0-5cca-45a2-b96f-ae16bb8985f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:16.194412 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a4fe0-5cca-45a2-b96f-ae16bb8985f9 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:16.194689 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a4fe0-5cca-45a2-b96f-ae16bb8985f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:16.194962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a4fe0-5cca-45a2-b96f-ae16bb8985f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:16.196192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53a4fe0-5cca-45a2-b96f-ae16bb8985f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:16.197706 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53a4fe0-5cca-45a2-b96f-ae16bb8985f9 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:16.199679 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 155/301] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:16.200526 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a4fe0-5cca-45a2-b96f-ae16bb8985f9 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:16.251377 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:16.254535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786a1e9-229a-438e-85e6-bc4ba1933322 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:16.271523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:16.292819 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b899c5e-986f-4fd1-bd74-9b7eff2dba28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:16.293481 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b899c5e-986f-4fd1-bd74-9b7eff2dba28 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:16.293765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b899c5e-986f-4fd1-bd74-9b7eff2dba28 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:16.294045 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b899c5e-986f-4fd1-bd74-9b7eff2dba28 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:16.307105 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b899c5e-986f-4fd1-bd74-9b7eff2dba28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:16.312213 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-9b899c5e-986f-4fd1-bd74-9b7eff2dba28 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:46:16.312213 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:16.312213 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:16.312213 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:16.312213 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:16.312213 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:16.312213 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:16.312213 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:16.312213 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:16.312213 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:16.312213 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:16.312213 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:16.312213 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:16.312213 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:16.312213 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:16.312213 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:16.313871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:16.313871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:16.313871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:16.313871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:16.313871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:16.313871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:16.313871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:16.313871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:16.313871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:16.313871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:16.313871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:16.313871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:16.313871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:16.313871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:16.313871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:16.313871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:16.315449 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:16.315449 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:16.315449 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:16.315449 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:16.315449 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:16.315449 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:16.315449 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:16.315449 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:16.315449 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:16.315449 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:16.315449 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:46:16.315449 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:16.315449 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 147/302] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:16.315449 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b899c5e-986f-4fd1-bd74-9b7eff2dba28 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:16.340503 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a4fe0-5cca-45a2-b96f-ae16bb8985f9 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:16.394800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da82f02-8ad3-4828-9175-cc74fcb1110a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:16.395396 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da82f02-8ad3-4828-9175-cc74fcb1110a None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:16.395716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da82f02-8ad3-4828-9175-cc74fcb1110a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:16.395993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da82f02-8ad3-4828-9175-cc74fcb1110a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:16.407618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da82f02-8ad3-4828-9175-cc74fcb1110a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:16.413163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-7da82f02-8ad3-4828-9175-cc74fcb1110a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 04:46:16.413163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:16.413163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:16.413163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:16.413163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:16.413163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:16.413163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:16.413163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:16.413163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:16.413163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:16.413163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:16.413163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:16.413163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:16.413163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:16.413163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:16.413163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:16.414097 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:16.414097 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:16.414097 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:16.414097 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:16.414097 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:16.414097 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:16.414097 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:16.414097 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:16.414097 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:16.414097 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:16.414097 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:16.414097 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:16.414097 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:16.414097 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:16.414097 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:16.414097 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:16.414916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:16.414916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:16.414916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:16.414916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:16.414916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:16.414916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:16.414916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:16.414916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:16.414916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:16.414916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:16.414916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 04:46:16.414916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:16.414916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 156/303] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:16.416073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da82f02-8ad3-4828-9175-cc74fcb1110a None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:16.462510 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b899c5e-986f-4fd1-bd74-9b7eff2dba28 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:16.503119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f89234-5e8c-4534-b4ed-ce21c8e904c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:16.503798 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f89234-5e8c-4534-b4ed-ce21c8e904c2 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:16.504082 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f89234-5e8c-4534-b4ed-ce21c8e904c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:16.504361 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f89234-5e8c-4534-b4ed-ce21c8e904c2 None admin] PATH_INFO: `/v3/groups` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:16.505044 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f89234-5e8c-4534-b4ed-ce21c8e904c2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:16.506531 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f89234-5e8c-4534-b4ed-ce21c8e904c2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:16.535588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 148/304] 10.222.0.17 () {60 vars in 1134 bytes} [Tue Jul 26 04:46:16 2022] POST /identity/v3/groups => generated 229 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:46:16.535588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f89234-5e8c-4534-b4ed-ce21c8e904c2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:16.535863 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da82f02-8ad3-4828-9175-cc74fcb1110a None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:16.540939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:16.557173 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:16.585394 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fa923-bf80-4d67-913e-44c41deed4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:16.586096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fa923-bf80-4d67-913e-44c41deed4ae None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:16.586383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fa923-bf80-4d67-913e-44c41deed4ae None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:16.586659 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fa923-bf80-4d67-913e-44c41deed4ae None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:16.587464 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15fa923-bf80-4d67-913e-44c41deed4ae None admin] RBAC: Adding query filter params (name=member) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:16.588510 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15fa923-bf80-4d67-913e-44c41deed4ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:16.590573 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15fa923-bf80-4d67-913e-44c41deed4ae None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:16.607758 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 157/305] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:46:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:16.611034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fa923-bf80-4d67-913e-44c41deed4ae None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:16.673129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f89234-5e8c-4534-b4ed-ce21c8e904c2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:16.714714 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be49d0b5-a30a-457c-b084-3e0a9204c8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:16.715634 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be49d0b5-a30a-457c-b084-3e0a9204c8d4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:16.715976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be49d0b5-a30a-457c-b084-3e0a9204c8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:16.716305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be49d0b5-a30a-457c-b084-3e0a9204c8d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:16.717134 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be49d0b5-a30a-457c-b084-3e0a9204c8d4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:16.717490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be49d0b5-a30a-457c-b084-3e0a9204c8d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:16.718951 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be49d0b5-a30a-457c-b084-3e0a9204c8d4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:16.728965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 149/306] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:46:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:16.734574 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be49d0b5-a30a-457c-b084-3e0a9204c8d4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:16.756489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fa923-bf80-4d67-913e-44c41deed4ae None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:16.798635 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d96bc-a74e-4a2a-b4f4-aca9e5afa9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:16.799429 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d96bc-a74e-4a2a-b4f4-aca9e5afa9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:16.799665 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d96bc-a74e-4a2a-b4f4-aca9e5afa9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:16.799870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d96bc-a74e-4a2a-b4f4-aca9e5afa9f4 None admin] PATH_INFO: `/v3/users/2f37b195dfb547c58a73c2bfed97b610` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:16.801085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d96bc-a74e-4a2a-b4f4-aca9e5afa9f4 None admin] RBAC: Authorizing `identity:get_user(user_id=2f37b195dfb547c58a73c2bfed97b610)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:16.802541 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d96bc-a74e-4a2a-b4f4-aca9e5afa9f4 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:16.804657 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 158/307] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:46:16 2022] GET /identity/v3/users/2f37b195dfb547c58a73c2bfed97b610 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:16.810874 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d96bc-a74e-4a2a-b4f4-aca9e5afa9f4 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:16.840623 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:16.869477 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:16.873862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be49d0b5-a30a-457c-b084-3e0a9204c8d4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:16.919092 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5a796-12a5-4e02-8d7e-2fcd32f4a04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:16.919792 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5a796-12a5-4e02-8d7e-2fcd32f4a04c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:16.920164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5a796-12a5-4e02-8d7e-2fcd32f4a04c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:16.920485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5a796-12a5-4e02-8d7e-2fcd32f4a04c None admin] PATH_INFO: `/v3/users/a7b7a60d2feb499bb8e2151f8a55049a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:16.939848 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5a796-12a5-4e02-8d7e-2fcd32f4a04c None admin] RBAC: Authorizing `identity:get_user(user_id=a7b7a60d2feb499bb8e2151f8a55049a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:16.941583 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5a796-12a5-4e02-8d7e-2fcd32f4a04c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:16.948865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 150/308] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:46:16 2022] GET /identity/v3/users/a7b7a60d2feb499bb8e2151f8a55049a => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:16.950674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5a796-12a5-4e02-8d7e-2fcd32f4a04c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:16.958999 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d96bc-a74e-4a2a-b4f4-aca9e5afa9f4 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:16.997901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea38925-e666-4e32-a926-c09f18d39cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:16.998682 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea38925-e666-4e32-a926-c09f18d39cf8 None admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:16.998969 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea38925-e666-4e32-a926-c09f18d39cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:16.999250 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea38925-e666-4e32-a926-c09f18d39cf8 None admin] PATH_INFO: `/v3/system/users/2f37b195dfb547c58a73c2bfed97b610/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:17.001182 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea38925-e666-4e32-a926-c09f18d39cf8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2f37b195dfb547c58a73c2bfed97b610, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:17.002986 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea38925-e666-4e32-a926-c09f18d39cf8 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:17.016197 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 159/309] 10.222.0.17 () {58 vars in 1334 bytes} [Tue Jul 26 04:46:16 2022] PUT /identity/v3/system/users/2f37b195dfb547c58a73c2bfed97b610/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 04:46:17.089465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5a796-12a5-4e02-8d7e-2fcd32f4a04c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:17.135106 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe6c72-61fe-4359-b9cc-572359734550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:17.136278 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe6c72-61fe-4359-b9cc-572359734550 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:17.136752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe6c72-61fe-4359-b9cc-572359734550 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:17.137209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe6c72-61fe-4359-b9cc-572359734550 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:17.138235 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe6c72-61fe-4359-b9cc-572359734550 None admin] RBAC: Adding query filter params (role.id=6c31395d654e4add9c982f62b79c2873, user.id=a7b7a60d2feb499bb8e2151f8a55049a, scope.system=all) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:17.138826 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe6c72-61fe-4359-b9cc-572359734550 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:17.140778 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe6c72-61fe-4359-b9cc-572359734550 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:17.147449 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 151/310] 10.222.0.17 () {56 vars in 1308 bytes} [Tue Jul 26 04:46:16 2022] GET /identity/v3/role_assignments?user.id=a7b7a60d2feb499bb8e2151f8a55049a&scope.system=all&role.id=6c31395d654e4add9c982f62b79c2873 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:17.494541 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37257e-99f6-40a5-a519-301f8d46ecf8 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:17.494922 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37257e-99f6-40a5-a519-301f8d46ecf8 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:17.495105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37257e-99f6-40a5-a519-301f8d46ecf8 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:17.496772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 160/311] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:17.503001 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191a81f-f470-4de4-86e4-a20d1d254ddd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:17.503281 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191a81f-f470-4de4-86e4-a20d1d254ddd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:17.503579 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191a81f-f470-4de4-86e4-a20d1d254ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:17.511851 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:17.523483 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d191a81f-f470-4de4-86e4-a20d1d254ddd None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:17.524988 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d191a81f-f470-4de4-86e4-a20d1d254ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:17.599865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 152/312] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:17.622919 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37257e-99f6-40a5-a519-301f8d46ecf8 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:17.737326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37257e-99f6-40a5-a519-301f8d46ecf8 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:17.739667 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:17.776409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b112a-2bab-4136-b785-f2ad6d98fa2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:17.777072 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b112a-2bab-4136-b785-f2ad6d98fa2e None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:17.777360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b112a-2bab-4136-b785-f2ad6d98fa2e None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:17.777636 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b112a-2bab-4136-b785-f2ad6d98fa2e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:17.788833 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204b112a-2bab-4136-b785-f2ad6d98fa2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:17.793989 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-204b112a-2bab-4136-b785-f2ad6d98fa2e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 04:46:17.793989 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:17.793989 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:17.793989 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:17.793989 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:17.793989 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:17.793989 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:17.793989 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:17.793989 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:17.793989 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:17.793989 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:17.793989 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:17.793989 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:17.793989 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:17.793989 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:17.793989 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:17.794977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:17.794977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:17.794977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:17.794977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:17.794977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:17.794977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:17.794977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:17.794977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:17.794977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:17.794977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:17.794977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:17.794977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:17.794977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:17.794977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:17.794977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:17.794977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:17.795843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:17.795843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:17.795843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:17.795843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:17.795843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:17.795843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:17.795843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:17.795843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:17.795843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:17.795843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:17.795843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 04:46:17.795843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:17.795843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 161/313] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:17.803300 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191a81f-f470-4de4-86e4-a20d1d254ddd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:17.908313 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191a81f-f470-4de4-86e4-a20d1d254ddd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:17.947831 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826fa53-a3c2-4b40-a9f3-77e13d409607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:17.948466 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826fa53-a3c2-4b40-a9f3-77e13d409607 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:17.948749 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826fa53-a3c2-4b40-a9f3-77e13d409607 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:17.949023 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826fa53-a3c2-4b40-a9f3-77e13d409607 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:17.949747 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a826fa53-a3c2-4b40-a9f3-77e13d409607 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:17.950052 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a826fa53-a3c2-4b40-a9f3-77e13d409607 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:17.951483 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a826fa53-a3c2-4b40-a9f3-77e13d409607 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:17.952887 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:17.960565 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 153/314] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:46:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:17.967146 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b112a-2bab-4136-b785-f2ad6d98fa2e None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:18.072769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b112a-2bab-4136-b785-f2ad6d98fa2e None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:18.111721 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08778f-175c-4a24-a7e8-01e204c9f457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:18.112380 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08778f-175c-4a24-a7e8-01e204c9f457 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:18.112664 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08778f-175c-4a24-a7e8-01e204c9f457 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:18.112939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08778f-175c-4a24-a7e8-01e204c9f457 None admin] PATH_INFO: `/v3/users/e1d7022163484a0082d4869a9b9d6b6d` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:18.114122 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e08778f-175c-4a24-a7e8-01e204c9f457 None admin] RBAC: Authorizing `identity:get_user(user_id=e1d7022163484a0082d4869a9b9d6b6d)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:18.115617 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e08778f-175c-4a24-a7e8-01e204c9f457 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:18.117607 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 162/315] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:46:17 2022] GET /identity/v3/users/e1d7022163484a0082d4869a9b9d6b6d => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:18.123248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826fa53-a3c2-4b40-a9f3-77e13d409607 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:18.236205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826fa53-a3c2-4b40-a9f3-77e13d409607 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:18.270177 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662fef0-19b7-43fd-9c28-dfa2f4cedadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:18.270819 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662fef0-19b7-43fd-9c28-dfa2f4cedadd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:18.271102 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662fef0-19b7-43fd-9c28-dfa2f4cedadd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:18.271381 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662fef0-19b7-43fd-9c28-dfa2f4cedadd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:18.339408 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:18.347250 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1662fef0-19b7-43fd-9c28-dfa2f4cedadd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:18.348892 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1662fef0-19b7-43fd-9c28-dfa2f4cedadd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:18.449877 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 154/316] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:46:18.455591 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08778f-175c-4a24-a7e8-01e204c9f457 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:18.488809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:18.559784 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08778f-175c-4a24-a7e8-01e204c9f457 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:18.594830 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52309dd8-2fad-427a-b1b9-014c3dfa5aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:18.595404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52309dd8-2fad-427a-b1b9-014c3dfa5aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:18.595721 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52309dd8-2fad-427a-b1b9-014c3dfa5aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:18.596000 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52309dd8-2fad-427a-b1b9-014c3dfa5aa0 None admin] PATH_INFO: `/v3/projects/fce7a22bc8ff438d90035074a2c73590` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:18.597242 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52309dd8-2fad-427a-b1b9-014c3dfa5aa0 None admin] RBAC: Authorizing `identity:get_project(project_id=fce7a22bc8ff438d90035074a2c73590)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:18.598718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52309dd8-2fad-427a-b1b9-014c3dfa5aa0 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:18.600773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 163/317] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:18 2022] GET /identity/v3/projects/fce7a22bc8ff438d90035074a2c73590 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:18.606714 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662fef0-19b7-43fd-9c28-dfa2f4cedadd None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:18.705376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:18.714288 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662fef0-19b7-43fd-9c28-dfa2f4cedadd None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:18.756458 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7a872-bcbd-405f-84d2-730fc02e9a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:18.757107 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7a872-bcbd-405f-84d2-730fc02e9a27 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:18.757385 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7a872-bcbd-405f-84d2-730fc02e9a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:18.757663 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7a872-bcbd-405f-84d2-730fc02e9a27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:18.758564 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7a872-bcbd-405f-84d2-730fc02e9a27 None admin] RBAC: Adding query filter params (role.id=6c31395d654e4add9c982f62b79c2873, scope.project.id=fce7a22bc8ff438d90035074a2c73590, user.id=e1d7022163484a0082d4869a9b9d6b6d) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:18.758877 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7a872-bcbd-405f-84d2-730fc02e9a27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:18.760529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7a872-bcbd-405f-84d2-730fc02e9a27 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:18.767705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 155/318] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:46:18 2022] GET /identity/v3/role_assignments?user.id=e1d7022163484a0082d4869a9b9d6b6d&scope.project.id=fce7a22bc8ff438d90035074a2c73590&role.id=6c31395d654e4add9c982f62b79c2873 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:18.791931 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a49d30-611e-4a45-b82a-81dfdf53a4d5 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:18.792269 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a49d30-611e-4a45-b82a-81dfdf53a4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:18.792506 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a49d30-611e-4a45-b82a-81dfdf53a4d5 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:18.794332 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 164/319] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:18.800554 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72c2d2-f10b-4154-990f-d92f2478f772 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:18.800871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72c2d2-f10b-4154-990f-d92f2478f772 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:18.801116 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72c2d2-f10b-4154-990f-d92f2478f772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:18.821199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6a72c2d2-f10b-4154-990f-d92f2478f772 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:18.822455 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6a72c2d2-f10b-4154-990f-d92f2478f772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:18.899275 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 156/320] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:18.922589 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a49d30-611e-4a45-b82a-81dfdf53a4d5 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:19.038980 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:19.053711 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a49d30-611e-4a45-b82a-81dfdf53a4d5 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:19.074726 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c170c70-65c4-4522-ac06-81a423ed50c0 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:19.075317 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c170c70-65c4-4522-ac06-81a423ed50c0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:19.075317 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c170c70-65c4-4522-ac06-81a423ed50c0 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:19.077018 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 157/321] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:46:19.084584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01460da9-7a0c-428f-aa46-ab5fbd41e768 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:19.084875 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01460da9-7a0c-428f-aa46-ab5fbd41e768 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:19.085161 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01460da9-7a0c-428f-aa46-ab5fbd41e768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:19.096425 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da180e6d-1c9d-40f4-a9bc-4f9c2f7b6f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:19.097200 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da180e6d-1c9d-40f4-a9bc-4f9c2f7b6f89 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:19.097642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da180e6d-1c9d-40f4-a9bc-4f9c2f7b6f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:19.098054 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da180e6d-1c9d-40f4-a9bc-4f9c2f7b6f89 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:19.099956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da180e6d-1c9d-40f4-a9bc-4f9c2f7b6f89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:19.101765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da180e6d-1c9d-40f4-a9bc-4f9c2f7b6f89 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:19.107617 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 165/322] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:19.107879 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-01460da9-7a0c-428f-aa46-ab5fbd41e768 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:19.107879 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-01460da9-7a0c-428f-aa46-ab5fbd41e768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:19.110274 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da180e6d-1c9d-40f4-a9bc-4f9c2f7b6f89 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:19.193378 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 158/323] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 04:46:19.209540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01460da9-7a0c-428f-aa46-ab5fbd41e768 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:19.223972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da180e6d-1c9d-40f4-a9bc-4f9c2f7b6f89 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:19.234399 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:19.260086 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06425fb4-bab3-4af3-90a9-25b18ad0e340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:19.260731 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06425fb4-bab3-4af3-90a9-25b18ad0e340 None admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:19.261010 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06425fb4-bab3-4af3-90a9-25b18ad0e340 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:19.261276 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06425fb4-bab3-4af3-90a9-25b18ad0e340 None admin] PATH_INFO: `/v3/groups` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:19.261971 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06425fb4-bab3-4af3-90a9-25b18ad0e340 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:19.263456 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06425fb4-bab3-4af3-90a9-25b18ad0e340 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:19.284546 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 166/324] 10.222.0.17 () {60 vars in 1134 bytes} [Tue Jul 26 04:46:19 2022] POST /identity/v3/groups => generated 226 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:46:19.320747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ded94-6a46-478f-b7c5-a630960bec18 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:19.321036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ded94-6a46-478f-b7c5-a630960bec18 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:19.321296 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ded94-6a46-478f-b7c5-a630960bec18 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:19.327255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 167/325] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:46:19.329287 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbbf4b-1783-4263-b4b2-9214a9559cdf None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:19.329577 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbbf4b-1783-4263-b4b2-9214a9559cdf None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:19.329858 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbbf4b-1783-4263-b4b2-9214a9559cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:19.333844 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01460da9-7a0c-428f-aa46-ab5fbd41e768 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:19.354321 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-34cbbf4b-1783-4263-b4b2-9214a9559cdf None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:19.355621 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-34cbbf4b-1783-4263-b4b2-9214a9559cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:19.382897 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:19.386771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4286a5-9864-4c70-b72a-7f1b0876a55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:19.386771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4286a5-9864-4c70-b72a-7f1b0876a55a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:19.386771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4286a5-9864-4c70-b72a-7f1b0876a55a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:19.386771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4286a5-9864-4c70-b72a-7f1b0876a55a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:19.401614 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4286a5-9864-4c70-b72a-7f1b0876a55a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:19.408469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-af4286a5-9864-4c70-b72a-7f1b0876a55a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:46:19.408469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:19.408469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:19.408469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:19.408469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:19.408469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:19.408469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:19.408469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:19.408469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:19.408469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:19.408469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:19.408469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:19.408469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:19.408469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:19.408469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:19.408469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:19.409673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:19.409673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:19.409673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:19.409673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:19.409673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:19.409673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:19.409673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:19.409673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:19.409673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:19.409673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:19.409673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:19.409673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:19.409673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:19.409673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:19.409673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:19.409673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:19.419098 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:19.419098 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:19.419098 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:19.419098 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:19.419098 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:19.419098 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:19.419098 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:19.419098 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:19.419098 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:19.419098 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:19.419098 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:46:19.419098 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:19.419098 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 159/326] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:46:19.420650 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4286a5-9864-4c70-b72a-7f1b0876a55a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:19.465606 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 168/327] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:19.488635 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbbf4b-1783-4263-b4b2-9214a9559cdf None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:19.527633 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:19.549113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4286a5-9864-4c70-b72a-7f1b0876a55a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:19.601217 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8cbe3-28fd-4823-8101-b2952fa8c8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:19.602126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8cbe3-28fd-4823-8101-b2952fa8c8af None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:19.602716 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8cbe3-28fd-4823-8101-b2952fa8c8af None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:19.603169 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8cbe3-28fd-4823-8101-b2952fa8c8af None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:19.604836 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8cbe3-28fd-4823-8101-b2952fa8c8af None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:19.605314 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8cbe3-28fd-4823-8101-b2952fa8c8af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:19.607750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8cbe3-28fd-4823-8101-b2952fa8c8af None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:19.625561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbbf4b-1783-4263-b4b2-9214a9559cdf None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:19.626035 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 160/328] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:46:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:19.629877 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8cbe3-28fd-4823-8101-b2952fa8c8af None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:19.666714 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32abc08-051f-4a03-8244-3b4c9fdbf2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:19.667569 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32abc08-051f-4a03-8244-3b4c9fdbf2ed None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:19.668010 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32abc08-051f-4a03-8244-3b4c9fdbf2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:19.668744 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32abc08-051f-4a03-8244-3b4c9fdbf2ed None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:19.687853 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32abc08-051f-4a03-8244-3b4c9fdbf2ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:19.692902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-b32abc08-051f-4a03-8244-3b4c9fdbf2ed None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 04:46:19.692902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:19.692902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:19.692902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:19.692902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:19.692902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:19.692902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:19.692902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:19.692902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:19.692902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:19.692902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:19.692902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:19.692902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:19.692902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:19.692902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:19.692902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:19.694137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:19.694137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:19.694137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:19.694137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:19.694137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:19.694137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:19.694137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:19.694137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:19.694137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:19.694137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:19.694137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:19.694137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:19.694137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:19.694137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:19.694137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:19.694137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:19.694941 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:19.694941 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:19.694941 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:19.694941 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:19.694941 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:19.694941 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:19.694941 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:19.694941 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:19.694941 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:19.694941 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:19.694941 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 04:46:19.694941 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:19.694941 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 169/329] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:46:19.700970 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32abc08-051f-4a03-8244-3b4c9fdbf2ed None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:19.731672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:19.783066 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8cbe3-28fd-4823-8101-b2952fa8c8af None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:19.845366 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854db8ac-3777-49c0-9df2-e9943883e685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:19.846135 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854db8ac-3777-49c0-9df2-e9943883e685 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:19.846422 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854db8ac-3777-49c0-9df2-e9943883e685 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:19.846699 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854db8ac-3777-49c0-9df2-e9943883e685 None admin] PATH_INFO: `/v3/users/2f37b195dfb547c58a73c2bfed97b610` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:19.846975 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32abc08-051f-4a03-8244-3b4c9fdbf2ed None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:19.848080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854db8ac-3777-49c0-9df2-e9943883e685 None admin] RBAC: Authorizing `identity:get_user(user_id=2f37b195dfb547c58a73c2bfed97b610)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:19.851014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854db8ac-3777-49c0-9df2-e9943883e685 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:19.853239 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 161/330] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:46:19 2022] GET /identity/v3/users/2f37b195dfb547c58a73c2bfed97b610 => generated 293 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:19.859429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854db8ac-3777-49c0-9df2-e9943883e685 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:19.876949 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:19.896626 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd64e9-a920-47ac-9609-4fd576a21aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:19.897264 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd64e9-a920-47ac-9609-4fd576a21aed None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:19.897542 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd64e9-a920-47ac-9609-4fd576a21aed None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:19.897811 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd64e9-a920-47ac-9609-4fd576a21aed None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:19.898545 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd64e9-a920-47ac-9609-4fd576a21aed None admin] RBAC: Adding query filter params (name=reader) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:19.898850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd64e9-a920-47ac-9609-4fd576a21aed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:19.900293 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd64e9-a920-47ac-9609-4fd576a21aed None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:19.915684 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 170/331] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:46:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:19.919280 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd64e9-a920-47ac-9609-4fd576a21aed None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:19.993956 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854db8ac-3777-49c0-9df2-e9943883e685 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:20.021100 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:20.042229 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce0d8d-fe15-4f85-b07a-f56f10436042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:20.042439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd64e9-a920-47ac-9609-4fd576a21aed None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:20.043008 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce0d8d-fe15-4f85-b07a-f56f10436042 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:20.043211 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce0d8d-fe15-4f85-b07a-f56f10436042 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:20.043490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce0d8d-fe15-4f85-b07a-f56f10436042 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:20.045680 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce0d8d-fe15-4f85-b07a-f56f10436042 None admin] RBAC: Adding query filter params (role.id=7519356859fb40508c25bec8161207fd, user.id=2f37b195dfb547c58a73c2bfed97b610, scope.system=all) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:20.046007 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce0d8d-fe15-4f85-b07a-f56f10436042 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:20.047470 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce0d8d-fe15-4f85-b07a-f56f10436042 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:20.066178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 162/332] 10.222.0.17 () {56 vars in 1308 bytes} [Tue Jul 26 04:46:19 2022] GET /identity/v3/role_assignments?user.id=2f37b195dfb547c58a73c2bfed97b610&scope.system=all&role.id=7519356859fb40508c25bec8161207fd => generated 510 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:20.108229 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b43d0-915c-4a7a-bd72-9967382637e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:20.108921 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b43d0-915c-4a7a-bd72-9967382637e6 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:20.109210 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b43d0-915c-4a7a-bd72-9967382637e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:20.109489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b43d0-915c-4a7a-bd72-9967382637e6 None admin] PATH_INFO: `/v3/users/a7b7a60d2feb499bb8e2151f8a55049a` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:20.111658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b43d0-915c-4a7a-bd72-9967382637e6 None admin] RBAC: Authorizing `identity:get_user(user_id=a7b7a60d2feb499bb8e2151f8a55049a)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:20.113183 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b43d0-915c-4a7a-bd72-9967382637e6 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:20.115352 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 171/333] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:46:19 2022] GET /identity/v3/users/a7b7a60d2feb499bb8e2151f8a55049a => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:20.121384 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce0d8d-fe15-4f85-b07a-f56f10436042 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:20.213327 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:20.253379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce0d8d-fe15-4f85-b07a-f56f10436042 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:20.291859 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c1429-7c56-48e8-b9b8-e787b25e3b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:20.292556 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c1429-7c56-48e8-b9b8-e787b25e3b1f None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:20.292841 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c1429-7c56-48e8-b9b8-e787b25e3b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:20.293121 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c1429-7c56-48e8-b9b8-e787b25e3b1f None admin] PATH_INFO: `/v3/system/users/a7b7a60d2feb499bb8e2151f8a55049a/roles/6c31395d654e4add9c982f62b79c2873` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:20.294635 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c1429-7c56-48e8-b9b8-e787b25e3b1f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a7b7a60d2feb499bb8e2151f8a55049a, role_id=6c31395d654e4add9c982f62b79c2873)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:20.296185 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c1429-7c56-48e8-b9b8-e787b25e3b1f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:20.306677 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 163/334] 10.222.0.17 () {58 vars in 1334 bytes} [Tue Jul 26 04:46:20 2022] PUT /identity/v3/system/users/a7b7a60d2feb499bb8e2151f8a55049a/roles/6c31395d654e4add9c982f62b79c2873 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:46:22.066440 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5db24-d1dc-4501-a2ee-f92645732bc2 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:22.067156 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5db24-d1dc-4501-a2ee-f92645732bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:22.067156 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5db24-d1dc-4501-a2ee-f92645732bc2 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:22.068705 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 172/335] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:22.082140 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b16ff5-3fbd-48f5-a63e-2c703612bd84 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:22.082807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b16ff5-3fbd-48f5-a63e-2c703612bd84 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:22.082807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b16ff5-3fbd-48f5-a63e-2c703612bd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:22.112753 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-55b16ff5-3fbd-48f5-a63e-2c703612bd84 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:22.113989 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-55b16ff5-3fbd-48f5-a63e-2c703612bd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:22.202888 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 164/336] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:22.234355 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec858bfa-e265-44ce-9321-b5ed093371db None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:22.234986 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec858bfa-e265-44ce-9321-b5ed093371db None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:22.235471 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec858bfa-e265-44ce-9321-b5ed093371db None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:22.237344 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 173/337] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:22.244132 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4aa449-32df-4c7c-8249-2ea29d0c3b7e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:22.244402 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4aa449-32df-4c7c-8249-2ea29d0c3b7e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:22.244685 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4aa449-32df-4c7c-8249-2ea29d0c3b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:22.255106 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec858bfa-e265-44ce-9321-b5ed093371db None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:22.273848 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4c4aa449-32df-4c7c-8249-2ea29d0c3b7e None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:22.275085 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4c4aa449-32df-4c7c-8249-2ea29d0c3b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:22.394658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec858bfa-e265-44ce-9321-b5ed093371db None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:22.419631 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:22.430143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 165/338] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:22.454113 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71a696-1ed9-4204-b7de-3b1316e56b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:22.455147 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71a696-1ed9-4204-b7de-3b1316e56b9d None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:22.455688 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71a696-1ed9-4204-b7de-3b1316e56b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:22.456163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71a696-1ed9-4204-b7de-3b1316e56b9d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:22.471631 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b71a696-1ed9-4204-b7de-3b1316e56b9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:22.477053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-6b71a696-1ed9-4204-b7de-3b1316e56b9d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 04:46:22.477053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:22.477053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:22.477053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:22.477053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:22.477053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:22.477053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:22.477053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:22.477053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:22.477053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:22.477053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:22.477053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:22.477053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:22.477053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:22.477053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:22.477053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:22.477936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:22.477936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:22.477936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:22.477936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:22.477936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:22.477936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:22.477936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:22.477936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:22.477936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:22.477936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:22.477936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:22.477936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:22.477936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:22.477936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:22.477936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:22.477936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:22.478726 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:22.478726 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:22.478726 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:22.478726 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:22.478726 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:22.478726 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:22.478726 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:22.478726 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:22.478726 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:22.478726 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:22.478726 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 04:46:22.478726 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:22.483438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4aa449-32df-4c7c-8249-2ea29d0c3b7e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:22.483746 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 174/339] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jul 26 04:46:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:22.489755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71a696-1ed9-4204-b7de-3b1316e56b9d None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:22.505106 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:22.616211 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71a696-1ed9-4204-b7de-3b1316e56b9d None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:22.655243 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90808d1b-0982-486b-8d9d-d9cbb513cd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:22.656388 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90808d1b-0982-486b-8d9d-d9cbb513cd1b None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:22.656925 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90808d1b-0982-486b-8d9d-d9cbb513cd1b None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:22.657400 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90808d1b-0982-486b-8d9d-d9cbb513cd1b None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:22.659045 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90808d1b-0982-486b-8d9d-d9cbb513cd1b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:22.659624 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90808d1b-0982-486b-8d9d-d9cbb513cd1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:22.661279 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90808d1b-0982-486b-8d9d-d9cbb513cd1b None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:22.672340 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 175/340] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:46:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:22.678924 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90808d1b-0982-486b-8d9d-d9cbb513cd1b None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:22.710299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4aa449-32df-4c7c-8249-2ea29d0c3b7e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:22.765003 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:22.792898 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90808d1b-0982-486b-8d9d-d9cbb513cd1b None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:22.812763 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2082faa-a760-4689-ab4d-4551886cd0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:22.813447 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2082faa-a760-4689-ab4d-4551886cd0f1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:22.813730 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2082faa-a760-4689-ab4d-4551886cd0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:22.814005 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2082faa-a760-4689-ab4d-4551886cd0f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:22.830901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490afde-2677-453a-aef0-ffed933b1441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:22.832166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490afde-2677-453a-aef0-ffed933b1441 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:22.832652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490afde-2677-453a-aef0-ffed933b1441 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:22.833341 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490afde-2677-453a-aef0-ffed933b1441 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:22.844951 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2082faa-a760-4689-ab4d-4551886cd0f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:22.850267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d2082faa-a760-4689-ab4d-4551886cd0f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:46:22.850267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:22.850267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:22.850267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:22.850267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:22.850267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:22.850267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:22.850267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:22.850267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:22.850267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:22.850267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:22.850267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:22.850267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:22.850267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:22.850267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:22.850267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:22.851191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:22.851191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:22.851191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:22.851191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:22.851191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:22.851191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:22.851191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:22.851191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:22.851191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:22.851191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:22.851191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:22.851191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:22.851191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:22.851191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:22.851191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:22.851191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:22.852025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:22.852025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:22.852025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:22.852025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:22.852025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:22.852025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:22.852025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:22.852025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:22.852025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:22.852025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:22.852025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:46:22.852025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:22.852025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 166/341] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:46:22.860676 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2082faa-a760-4689-ab4d-4551886cd0f1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:22.928751 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490afde-2677-453a-aef0-ffed933b1441 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:22.930608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490afde-2677-453a-aef0-ffed933b1441 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:22.970704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:23.030168 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2082faa-a760-4689-ab4d-4551886cd0f1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:23.039586 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:23.078231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 176/342] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:46:23.084207 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c60f91-7778-4459-9d0b-21e498206f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:23.084869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c60f91-7778-4459-9d0b-21e498206f57 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:23.085064 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490afde-2677-453a-aef0-ffed933b1441 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:23.085181 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c60f91-7778-4459-9d0b-21e498206f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:23.085402 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c60f91-7778-4459-9d0b-21e498206f57 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:23.086162 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c60f91-7778-4459-9d0b-21e498206f57 None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:23.086461 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c60f91-7778-4459-9d0b-21e498206f57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:23.087889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c60f91-7778-4459-9d0b-21e498206f57 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:23.107280 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 167/343] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:46:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:23.113048 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c60f91-7778-4459-9d0b-21e498206f57 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:23.200963 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490afde-2677-453a-aef0-ffed933b1441 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:23.240256 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a99507-ff09-48df-a566-98bf04c4fb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:23.241195 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a99507-ff09-48df-a566-98bf04c4fb04 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:23.241639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a99507-ff09-48df-a566-98bf04c4fb04 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:23.242065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a99507-ff09-48df-a566-98bf04c4fb04 None admin] PATH_INFO: `/v3/projects/8be3583b9359412ba4129e556beef77c` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:23.243482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a99507-ff09-48df-a566-98bf04c4fb04 None admin] RBAC: Authorizing `identity:get_project(project_id=8be3583b9359412ba4129e556beef77c)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:23.245186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a99507-ff09-48df-a566-98bf04c4fb04 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:23.247428 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 177/344] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:23 2022] GET /identity/v3/projects/8be3583b9359412ba4129e556beef77c => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:23.253256 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a99507-ff09-48df-a566-98bf04c4fb04 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:23.292183 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c60f91-7778-4459-9d0b-21e498206f57 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:23.301448 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:23.311316 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:23.364000 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8af11-06b5-460d-8386-c70d7226f57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:23.364685 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8af11-06b5-460d-8386-c70d7226f57d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:23.364928 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8af11-06b5-460d-8386-c70d7226f57d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:23.365197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8af11-06b5-460d-8386-c70d7226f57d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:23.368877 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a99507-ff09-48df-a566-98bf04c4fb04 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:23.406707 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cca489-3fb0-4f27-9845-51bdad7719a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:23.407675 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cca489-3fb0-4f27-9845-51bdad7719a4 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:23.408205 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cca489-3fb0-4f27-9845-51bdad7719a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:23.408679 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cca489-3fb0-4f27-9845-51bdad7719a4 None admin] PATH_INFO: `/v3/groups/a5b3ed6cc57044e19dfae5d635209526` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:23.415849 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cca489-3fb0-4f27-9845-51bdad7719a4 None admin] RBAC: Authorizing `identity:get_group(group_id=a5b3ed6cc57044e19dfae5d635209526)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:23.417584 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cca489-3fb0-4f27-9845-51bdad7719a4 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:23.419502 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 178/345] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jul 26 04:46:23 2022] GET /identity/v3/groups/a5b3ed6cc57044e19dfae5d635209526 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:23.425841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cca489-3fb0-4f27-9845-51bdad7719a4 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:23.465901 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8af11-06b5-460d-8386-c70d7226f57d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:23.467356 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8af11-06b5-460d-8386-c70d7226f57d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:23.540686 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cca489-3fb0-4f27-9845-51bdad7719a4 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:23.544029 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:23.579110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9a50a-444e-4590-9304-2cc5d271b40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:23.580130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9a50a-444e-4590-9304-2cc5d271b40f None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:23.580622 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9a50a-444e-4590-9304-2cc5d271b40f None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:23.581081 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9a50a-444e-4590-9304-2cc5d271b40f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:23.581895 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 168/346] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:46:23.582278 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa9a50a-444e-4590-9304-2cc5d271b40f None admin] RBAC: Adding query filter params (group.id=a5b3ed6cc57044e19dfae5d635209526, role.id=9223a9bb40484faca3a7bcfeb41a2215, scope.project.id=8be3583b9359412ba4129e556beef77c) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:23.582772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa9a50a-444e-4590-9304-2cc5d271b40f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:23.583213 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4e664-df1d-4788-9ebc-48dea4bdd1ac None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:23.583497 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4e664-df1d-4788-9ebc-48dea4bdd1ac None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:23.584928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa9a50a-444e-4590-9304-2cc5d271b40f None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:23.587947 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4e664-df1d-4788-9ebc-48dea4bdd1ac None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:23.589725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 169/347] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:23.591236 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4e664-df1d-4788-9ebc-48dea4bdd1ac None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:23.592816 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 179/348] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Jul 26 04:46:23 2022] GET /identity/v3/role_assignments?group.id=a5b3ed6cc57044e19dfae5d635209526&scope.project.id=8be3583b9359412ba4129e556beef77c&role.id=9223a9bb40484faca3a7bcfeb41a2215 => generated 260 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:23.601421 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d6420-ff01-4394-9e06-75fb93479d45 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:23.602081 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d6420-ff01-4394-9e06-75fb93479d45 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:23.602561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d6420-ff01-4394-9e06-75fb93479d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:23.610297 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:23.636880 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-a28d6420-ff01-4394-9e06-75fb93479d45 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:23.638137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a28d6420-ff01-4394-9e06-75fb93479d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:23.796402 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4e664-df1d-4788-9ebc-48dea4bdd1ac None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:23.805462 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 180/349] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:23.822073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337aacc-9517-4567-966d-ee8941a649f8 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:23.822386 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337aacc-9517-4567-966d-ee8941a649f8 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:23.822625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337aacc-9517-4567-966d-ee8941a649f8 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:23.824035 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 181/350] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:46:23.835739 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5b58d-90d8-435b-89af-e7ecec7c1bb5 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:23.836069 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5b58d-90d8-435b-89af-e7ecec7c1bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:23.836272 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5b58d-90d8-435b-89af-e7ecec7c1bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:23.841326 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c6971-d3df-4112-8fda-b84f2c467240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:23.842419 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c6971-d3df-4112-8fda-b84f2c467240 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:23.842909 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c6971-d3df-4112-8fda-b84f2c467240 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:23.843476 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c6971-d3df-4112-8fda-b84f2c467240 None admin] PATH_INFO: `/v3/projects/8be3583b9359412ba4129e556beef77c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:23.845277 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c6971-d3df-4112-8fda-b84f2c467240 None admin] RBAC: Authorizing `identity:get_project(project_id=8be3583b9359412ba4129e556beef77c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:23.846977 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c6971-d3df-4112-8fda-b84f2c467240 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:23.849245 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 170/351] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:23 2022] GET /identity/v3/projects/8be3583b9359412ba4129e556beef77c => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:23.850281 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c6971-d3df-4112-8fda-b84f2c467240 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:23.869022 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-dec5b58d-90d8-435b-89af-e7ecec7c1bb5 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:23.870253 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-dec5b58d-90d8-435b-89af-e7ecec7c1bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:23.940271 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:23.954661 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 182/352] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:23.955540 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5b58d-90d8-435b-89af-e7ecec7c1bb5 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:23.962833 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c6971-d3df-4112-8fda-b84f2c467240 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:23.986185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:24.004244 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228fa4c-7391-4122-bce7-2e732aa7aebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:24.005310 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228fa4c-7391-4122-bce7-2e732aa7aebe None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:24.005792 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228fa4c-7391-4122-bce7-2e732aa7aebe None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:24.006284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228fa4c-7391-4122-bce7-2e732aa7aebe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:24.018364 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d228fa4c-7391-4122-bce7-2e732aa7aebe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:24.024095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d228fa4c-7391-4122-bce7-2e732aa7aebe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:46:24.024095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:24.024095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:24.024095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:24.024095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:24.024095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:24.024095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:24.024095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:24.024095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:24.024095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:24.024095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:24.024095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:24.024095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:24.024095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:24.024095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:24.024095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:24.025083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:24.025083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:24.025083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:24.025083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:24.025083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:24.025083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:24.025083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:24.025083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:24.025083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:24.025083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:24.025083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:24.025083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:24.025083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:24.025083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:24.025083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:24.025083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:24.025936 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:24.025936 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:24.025936 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:24.025936 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:24.025936 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:24.025936 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:24.025936 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:24.025936 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:24.025936 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:24.025936 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:24.025936 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:46:24.025936 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:24.028484 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 171/353] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:24.030157 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24eb179-6771-4c99-aefe-fefc649f5320 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:24.030673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24eb179-6771-4c99-aefe-fefc649f5320 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:24.031152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24eb179-6771-4c99-aefe-fefc649f5320 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:24.032782 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 172/354] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:24.033710 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24eb179-6771-4c99-aefe-fefc649f5320 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:24.077267 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5b58d-90d8-435b-89af-e7ecec7c1bb5 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:24.117660 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ba060-7235-46e1-8339-8f258554a2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:24.118439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ba060-7235-46e1-8339-8f258554a2da None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:24.118643 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ba060-7235-46e1-8339-8f258554a2da None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:24.118922 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ba060-7235-46e1-8339-8f258554a2da None admin] PATH_INFO: `/v3/groups/a5b3ed6cc57044e19dfae5d635209526` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:24.120224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ba060-7235-46e1-8339-8f258554a2da None admin] RBAC: Authorizing `identity:get_group(group_id=a5b3ed6cc57044e19dfae5d635209526)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:24.121732 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ba060-7235-46e1-8339-8f258554a2da None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:24.128730 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 183/355] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jul 26 04:46:23 2022] GET /identity/v3/groups/a5b3ed6cc57044e19dfae5d635209526 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:24.129679 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ba060-7235-46e1-8339-8f258554a2da None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:24.149812 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24eb179-6771-4c99-aefe-fefc649f5320 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:24.187247 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21b8c7-b6ba-483e-aac5-83babc681014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:24.188315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21b8c7-b6ba-483e-aac5-83babc681014 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:24.188798 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21b8c7-b6ba-483e-aac5-83babc681014 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:24.189276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21b8c7-b6ba-483e-aac5-83babc681014 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:24.201148 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac21b8c7-b6ba-483e-aac5-83babc681014 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:24.206582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-ac21b8c7-b6ba-483e-aac5-83babc681014 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 04:46:24.206582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:24.206582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:24.206582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:24.206582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:24.206582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:24.206582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:24.206582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:24.206582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:24.206582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:24.206582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:24.206582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:24.206582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:24.206582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:24.206582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:24.206582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:24.207542 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:24.207542 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:24.207542 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:24.207542 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:24.207542 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:24.207542 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:24.207542 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:24.207542 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:24.207542 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:24.207542 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:24.207542 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:24.207542 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:24.207542 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:24.207542 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:24.207542 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:24.207542 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:24.208382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:24.208382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:24.208382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:24.208382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:24.208382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:24.208382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:24.208382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:24.208382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:24.208382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:24.208382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:24.208382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 04:46:24.208382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:24.209364 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 173/356] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:24.211415 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8264e-892e-40b1-8893-7c07edf44e87 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:24.211740 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8264e-892e-40b1-8893-7c07edf44e87 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:24.212022 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8264e-892e-40b1-8893-7c07edf44e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:24.227411 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:24.241267 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:24.247243 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c4e8264e-892e-40b1-8893-7c07edf44e87 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:24.248551 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c4e8264e-892e-40b1-8893-7c07edf44e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:24.253565 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ba060-7235-46e1-8339-8f258554a2da None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:24.293025 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8779e8-1d91-4fbc-a99f-6024216153f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:24.293712 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8779e8-1d91-4fbc-a99f-6024216153f8 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:24.293990 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8779e8-1d91-4fbc-a99f-6024216153f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:24.294264 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8779e8-1d91-4fbc-a99f-6024216153f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:24.295103 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8779e8-1d91-4fbc-a99f-6024216153f8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:24.295406 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8779e8-1d91-4fbc-a99f-6024216153f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:24.296856 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8779e8-1d91-4fbc-a99f-6024216153f8 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:24.306732 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 184/357] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:46:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:24.310406 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8779e8-1d91-4fbc-a99f-6024216153f8 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:24.338101 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 174/358] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:24.339941 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b746d1-44bc-40d2-ab52-e536c0d3d6fe None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:24.340236 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b746d1-44bc-40d2-ab52-e536c0d3d6fe None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:24.340519 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b746d1-44bc-40d2-ab52-e536c0d3d6fe None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:24.342675 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 175/359] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:24.343461 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b746d1-44bc-40d2-ab52-e536c0d3d6fe None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:24.432396 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8779e8-1d91-4fbc-a99f-6024216153f8 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:24.484651 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5e1ea-ff6b-4be3-a24e-615dc6203ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:24.485289 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5e1ea-ff6b-4be3-a24e-615dc6203ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:24.485569 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5e1ea-ff6b-4be3-a24e-615dc6203ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:24.485848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5e1ea-ff6b-4be3-a24e-615dc6203ce9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:24.486655 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a5e1ea-ff6b-4be3-a24e-615dc6203ce9 None admin] RBAC: Adding query filter params (group.id=a5b3ed6cc57044e19dfae5d635209526, role.id=7519356859fb40508c25bec8161207fd, scope.project.id=8be3583b9359412ba4129e556beef77c) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:24.486967 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a5e1ea-ff6b-4be3-a24e-615dc6203ce9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:24.488503 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a5e1ea-ff6b-4be3-a24e-615dc6203ce9 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:24.491373 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b746d1-44bc-40d2-ab52-e536c0d3d6fe None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:24.495509 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 185/360] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Jul 26 04:46:24 2022] GET /identity/v3/role_assignments?group.id=a5b3ed6cc57044e19dfae5d635209526&scope.project.id=8be3583b9359412ba4129e556beef77c&role.id=7519356859fb40508c25bec8161207fd => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:24.496478 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5e1ea-ff6b-4be3-a24e-615dc6203ce9 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:24.522993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:24.528334 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:24.534711 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fbe474-2ba6-4635-88ca-af4de1541526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:24.535376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fbe474-2ba6-4635-88ca-af4de1541526 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:24.535706 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fbe474-2ba6-4635-88ca-af4de1541526 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:24.535986 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fbe474-2ba6-4635-88ca-af4de1541526 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:24.536767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fbe474-2ba6-4635-88ca-af4de1541526 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:24.537058 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fbe474-2ba6-4635-88ca-af4de1541526 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:24.538516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fbe474-2ba6-4635-88ca-af4de1541526 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:24.550159 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 176/361] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:46:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:24.551637 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2f77e-c387-470e-a2f9-9484bf4b0e32 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:24.551901 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2f77e-c387-470e-a2f9-9484bf4b0e32 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:24.552179 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2f77e-c387-470e-a2f9-9484bf4b0e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:24.573252 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7ae2f77e-c387-470e-a2f9-9484bf4b0e32 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:24.574519 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7ae2f77e-c387-470e-a2f9-9484bf4b0e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:24.644661 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5e1ea-ff6b-4be3-a24e-615dc6203ce9 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:24.669687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 177/362] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:24.671012 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2f77e-c387-470e-a2f9-9484bf4b0e32 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:24.692988 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f94571-c2df-4951-8f85-06ccd3510df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:24.693638 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f94571-c2df-4951-8f85-06ccd3510df2 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:24.693912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f94571-c2df-4951-8f85-06ccd3510df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:24.694189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f94571-c2df-4951-8f85-06ccd3510df2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:24.788874 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f94571-c2df-4951-8f85-06ccd3510df2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:24.790439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f94571-c2df-4951-8f85-06ccd3510df2 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:24.816268 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:24.847938 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2f77e-c387-470e-a2f9-9484bf4b0e32 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:24.883389 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:24.906956 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb51661-4f88-4f8e-b00b-cd8566b978c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:24.908711 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb51661-4f88-4f8e-b00b-cd8566b978c9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:24.909221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb51661-4f88-4f8e-b00b-cd8566b978c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:24.909928 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb51661-4f88-4f8e-b00b-cd8566b978c9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:24.919842 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 186/363] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:46:24.920757 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f94571-c2df-4951-8f85-06ccd3510df2 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:24.929376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb51661-4f88-4f8e-b00b-cd8566b978c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:24.934825 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-bbb51661-4f88-4f8e-b00b-cd8566b978c9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 04:46:24.934825 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:24.934825 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:24.934825 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:24.934825 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:24.934825 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:24.934825 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:24.934825 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:24.934825 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:24.934825 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:24.934825 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:24.934825 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:24.934825 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:24.934825 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:24.934825 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:24.934825 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:24.937437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:24.937437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:24.937437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:24.937437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:24.937437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:24.937437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:24.937437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:24.937437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:24.937437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:24.937437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:24.937437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:24.937437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:24.937437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:24.937437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:24.937437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:24.937437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:24.938230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:24.938230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:24.938230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:24.938230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:24.938230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:24.938230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:24.938230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:24.938230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:24.938230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:24.938230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:24.938230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 04:46:24.938230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:24.938230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 178/364] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jul 26 04:46:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:24.938230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb51661-4f88-4f8e-b00b-cd8566b978c9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:25.069180 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb51661-4f88-4f8e-b00b-cd8566b978c9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:25.073555 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f94571-c2df-4951-8f85-06ccd3510df2 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:25.104231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:25.120932 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ba64e-c543-4bf2-a810-334e09aea08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:25.120932 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ba64e-c543-4bf2-a810-334e09aea08f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:25.121458 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ba64e-c543-4bf2-a810-334e09aea08f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:25.121828 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ba64e-c543-4bf2-a810-334e09aea08f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:25.125488 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6355f68-e8c3-4be9-833f-f02de39f2ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:25.126678 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6355f68-e8c3-4be9-833f-f02de39f2ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:25.127116 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6355f68-e8c3-4be9-833f-f02de39f2ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:25.127545 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6355f68-e8c3-4be9-833f-f02de39f2ca0 None admin] PATH_INFO: `/v3/users/a7b7a60d2feb499bb8e2151f8a55049a` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:25.129111 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6355f68-e8c3-4be9-833f-f02de39f2ca0 None admin] RBAC: Authorizing `identity:get_user(user_id=a7b7a60d2feb499bb8e2151f8a55049a)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:25.130732 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6355f68-e8c3-4be9-833f-f02de39f2ca0 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:25.132965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 187/365] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:46:24 2022] GET /identity/v3/users/a7b7a60d2feb499bb8e2151f8a55049a => generated 293 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:25.134034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6355f68-e8c3-4be9-833f-f02de39f2ca0 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:25.134579 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ba64e-c543-4bf2-a810-334e09aea08f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:25.141277 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-5b0ba64e-c543-4bf2-a810-334e09aea08f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:46:25.141277 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:25.141277 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:25.141277 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:25.141277 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:25.141277 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:25.141277 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:25.141277 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:25.141277 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:25.141277 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:25.141277 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:25.141277 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:25.141277 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:25.141277 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:25.141277 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:25.141277 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:25.142434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:25.142434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:25.142434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:25.142434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:25.142434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:25.142434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:25.142434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:25.142434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:25.142434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:25.142434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:25.142434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:25.142434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:25.142434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:25.142434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:25.142434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:25.142434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:25.143386 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:25.143386 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:25.143386 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:25.143386 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:25.143386 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:25.143386 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:25.143386 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:25.143386 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:25.143386 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:25.143386 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:25.143386 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:46:25.143386 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:25.144888 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 179/366] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:46:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:25.145840 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ba64e-c543-4bf2-a810-334e09aea08f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:25.204965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:25.280756 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6355f68-e8c3-4be9-833f-f02de39f2ca0 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:25.286016 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ba64e-c543-4bf2-a810-334e09aea08f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:25.326131 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25fe8c-a624-46e7-b804-afded98d5988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:25.327515 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25fe8c-a624-46e7-b804-afded98d5988 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:25.328176 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25fe8c-a624-46e7-b804-afded98d5988 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:25.328675 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25fe8c-a624-46e7-b804-afded98d5988 None admin] PATH_INFO: `/v3/projects/fce7a22bc8ff438d90035074a2c73590` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:25.330446 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d25fe8c-a624-46e7-b804-afded98d5988 None admin] RBAC: Authorizing `identity:get_project(project_id=fce7a22bc8ff438d90035074a2c73590)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:25.333366 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d25fe8c-a624-46e7-b804-afded98d5988 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:25.336363 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 188/367] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:25 2022] GET /identity/v3/projects/fce7a22bc8ff438d90035074a2c73590 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:25.338658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25fe8c-a624-46e7-b804-afded98d5988 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:25.344259 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c325e-d480-40ee-b5be-dc85cc7d19f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:25.346086 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c325e-d480-40ee-b5be-dc85cc7d19f8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:25.348534 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c325e-d480-40ee-b5be-dc85cc7d19f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:25.348534 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c325e-d480-40ee-b5be-dc85cc7d19f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:25.348919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c325e-d480-40ee-b5be-dc85cc7d19f8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:25.349199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c325e-d480-40ee-b5be-dc85cc7d19f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:25.350701 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c325e-d480-40ee-b5be-dc85cc7d19f8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:25.363271 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 180/368] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:46:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:25.364214 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c325e-d480-40ee-b5be-dc85cc7d19f8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:25.426865 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:25.487703 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25fe8c-a624-46e7-b804-afded98d5988 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:25.507445 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c325e-d480-40ee-b5be-dc85cc7d19f8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:25.518785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:25.531648 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e379c2-b41e-435f-a9ba-69597ac6d4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:25.532760 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e379c2-b41e-435f-a9ba-69597ac6d4d3 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:25.533250 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e379c2-b41e-435f-a9ba-69597ac6d4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:25.533791 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e379c2-b41e-435f-a9ba-69597ac6d4d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:25.534798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e379c2-b41e-435f-a9ba-69597ac6d4d3 None admin] RBAC: Adding query filter params (role.id=6c31395d654e4add9c982f62b79c2873, user.id=a7b7a60d2feb499bb8e2151f8a55049a, scope.system=all) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:25.535298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e379c2-b41e-435f-a9ba-69597ac6d4d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:25.537183 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e379c2-b41e-435f-a9ba-69597ac6d4d3 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:25.551830 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 189/369] 10.222.0.17 () {56 vars in 1308 bytes} [Tue Jul 26 04:46:25 2022] GET /identity/v3/role_assignments?user.id=a7b7a60d2feb499bb8e2151f8a55049a&scope.system=all&role.id=6c31395d654e4add9c982f62b79c2873 => generated 510 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:25.553362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e379c2-b41e-435f-a9ba-69597ac6d4d3 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:25.553818 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8ebd8-0613-4522-accb-3bf1c112c049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:25.554558 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8ebd8-0613-4522-accb-3bf1c112c049 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:25.554805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8ebd8-0613-4522-accb-3bf1c112c049 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:25.555087 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8ebd8-0613-4522-accb-3bf1c112c049 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:25.555858 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c8ebd8-0613-4522-accb-3bf1c112c049 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:25.556170 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c8ebd8-0613-4522-accb-3bf1c112c049 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:25.558010 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c8ebd8-0613-4522-accb-3bf1c112c049 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:25.575276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 181/370] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:25.576254 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8ebd8-0613-4522-accb-3bf1c112c049 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:25.708436 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e379c2-b41e-435f-a9ba-69597ac6d4d3 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:25.738155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8ebd8-0613-4522-accb-3bf1c112c049 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:25.744951 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:25.780720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046fb41c-95eb-40c7-b622-70d4d897cb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:25.781922 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fb41c-95eb-40c7-b622-70d4d897cb5c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:25.782427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fb41c-95eb-40c7-b622-70d4d897cb5c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:25.782893 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fb41c-95eb-40c7-b622-70d4d897cb5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:25.820352 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db1966-be81-4ecb-840f-cf25b759d481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:25.821036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db1966-be81-4ecb-840f-cf25b759d481 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:25.821319 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db1966-be81-4ecb-840f-cf25b759d481 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:25.821594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db1966-be81-4ecb-840f-cf25b759d481 None admin] PATH_INFO: `/v3/groups/a5b3ed6cc57044e19dfae5d635209526` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:25.822846 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db1966-be81-4ecb-840f-cf25b759d481 None admin] RBAC: Authorizing `identity:get_group(group_id=a5b3ed6cc57044e19dfae5d635209526)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:25.824376 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db1966-be81-4ecb-840f-cf25b759d481 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:25.825400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:25.826510 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 190/371] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jul 26 04:46:25 2022] GET /identity/v3/groups/a5b3ed6cc57044e19dfae5d635209526 => generated 226 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:25.827575 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db1966-be81-4ecb-840f-cf25b759d481 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:25.863452 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046fb41c-95eb-40c7-b622-70d4d897cb5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:25.865110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046fb41c-95eb-40c7-b622-70d4d897cb5c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:25.970802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db1966-be81-4ecb-840f-cf25b759d481 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:25.977708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 182/372] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:46:25.978719 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046fb41c-95eb-40c7-b622-70d4d897cb5c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:26.016240 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e75cee0-a75d-46ed-ad3b-595f1907e19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:26.017349 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75cee0-a75d-46ed-ad3b-595f1907e19d None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:26.017845 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75cee0-a75d-46ed-ad3b-595f1907e19d None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:26.018406 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75cee0-a75d-46ed-ad3b-595f1907e19d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:26.051646 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:26.083994 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:26.088703 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046fb41c-95eb-40c7-b622-70d4d897cb5c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:26.104171 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e75cee0-a75d-46ed-ad3b-595f1907e19d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:26.106516 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e75cee0-a75d-46ed-ad3b-595f1907e19d None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:26.126947 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c3246-298c-417f-92c9-08bfc9b1e975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:26.128311 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c3246-298c-417f-92c9-08bfc9b1e975 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:26.128311 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c3246-298c-417f-92c9-08bfc9b1e975 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:26.128311 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c3246-298c-417f-92c9-08bfc9b1e975 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:26.128930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c3246-298c-417f-92c9-08bfc9b1e975 None admin] RBAC: Adding query filter params (group.id=a5b3ed6cc57044e19dfae5d635209526, role.id=7519356859fb40508c25bec8161207fd, scope.project.id=fce7a22bc8ff438d90035074a2c73590) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:26.129237 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c3246-298c-417f-92c9-08bfc9b1e975 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:26.130705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c3246-298c-417f-92c9-08bfc9b1e975 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:26.137160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 183/373] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Jul 26 04:46:25 2022] GET /identity/v3/role_assignments?group.id=a5b3ed6cc57044e19dfae5d635209526&scope.project.id=fce7a22bc8ff438d90035074a2c73590&role.id=7519356859fb40508c25bec8161207fd => generated 260 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:26.138047 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c3246-298c-417f-92c9-08bfc9b1e975 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:26.229153 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 191/374] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:46:26.231453 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395be2d5-c573-4f8f-be63-9c460ed6307d None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:26.232049 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395be2d5-c573-4f8f-be63-9c460ed6307d None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:26.232526 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395be2d5-c573-4f8f-be63-9c460ed6307d None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:26.234168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 192/375] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:26.235332 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395be2d5-c573-4f8f-be63-9c460ed6307d None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:26.253400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c3246-298c-417f-92c9-08bfc9b1e975 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:26.290209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:26.295380 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f69d58-42bb-463a-b992-cc13b427e6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:26.296681 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f69d58-42bb-463a-b992-cc13b427e6e7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:26.296681 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f69d58-42bb-463a-b992-cc13b427e6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:26.296681 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f69d58-42bb-463a-b992-cc13b427e6e7 None admin] PATH_INFO: `/v3/projects/fce7a22bc8ff438d90035074a2c73590` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:26.297884 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f69d58-42bb-463a-b992-cc13b427e6e7 None admin] RBAC: Authorizing `identity:get_project(project_id=fce7a22bc8ff438d90035074a2c73590)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:26.299478 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f69d58-42bb-463a-b992-cc13b427e6e7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:26.305315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 184/376] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:26 2022] GET /identity/v3/projects/fce7a22bc8ff438d90035074a2c73590 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:26.306947 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e02b7f-45a9-4727-81ec-6f64c01d4fe9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:26.307239 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e02b7f-45a9-4727-81ec-6f64c01d4fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:26.307517 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e02b7f-45a9-4727-81ec-6f64c01d4fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:26.331747 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-65e02b7f-45a9-4727-81ec-6f64c01d4fe9 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:26.331747 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-65e02b7f-45a9-4727-81ec-6f64c01d4fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:26.347663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:26.414530 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395be2d5-c573-4f8f-be63-9c460ed6307d None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:26.469862 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d2836-75b1-4c8f-ae10-864c9d35df49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:26.470457 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 185/377] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:26.470584 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d2836-75b1-4c8f-ae10-864c9d35df49 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:26.470798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d2836-75b1-4c8f-ae10-864c9d35df49 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:26.471068 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d2836-75b1-4c8f-ae10-864c9d35df49 None admin] PATH_INFO: `/v3/projects/d12823499382429e9825c57ea3292c56` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:26.471331 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e02b7f-45a9-4727-81ec-6f64c01d4fe9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:26.478725 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d2836-75b1-4c8f-ae10-864c9d35df49 None admin] RBAC: Authorizing `identity:get_project(project_id=d12823499382429e9825c57ea3292c56)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:26.480327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d2836-75b1-4c8f-ae10-864c9d35df49 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:26.482364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 193/378] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:26 2022] GET /identity/v3/projects/d12823499382429e9825c57ea3292c56 => generated 344 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:26.500888 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d2836-75b1-4c8f-ae10-864c9d35df49 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:26.638906 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d2836-75b1-4c8f-ae10-864c9d35df49 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:26.678248 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd9959-4065-4096-9407-a1f3a3a870c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:26.678923 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd9959-4065-4096-9407-a1f3a3a870c7 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:26.679209 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd9959-4065-4096-9407-a1f3a3a870c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:26.679489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd9959-4065-4096-9407-a1f3a3a870c7 None admin] PATH_INFO: `/v3/groups/d28be20a3a1441e59d04b97496814a5a` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:26.692135 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd9959-4065-4096-9407-a1f3a3a870c7 None admin] RBAC: Authorizing `identity:get_group(group_id=d28be20a3a1441e59d04b97496814a5a)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:26.693648 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd9959-4065-4096-9407-a1f3a3a870c7 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:26.695588 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 194/379] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jul 26 04:46:26 2022] GET /identity/v3/groups/d28be20a3a1441e59d04b97496814a5a => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:26.699581 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd9959-4065-4096-9407-a1f3a3a870c7 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:26.719775 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e02b7f-45a9-4727-81ec-6f64c01d4fe9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:26.738370 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:26.780929 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:26.796301 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae716ab-7c80-4175-8448-5b2115090cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:26.797246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae716ab-7c80-4175-8448-5b2115090cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:26.798717 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae716ab-7c80-4175-8448-5b2115090cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:26.799001 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae716ab-7c80-4175-8448-5b2115090cf6 None admin] PATH_INFO: `/v3/groups/a5b3ed6cc57044e19dfae5d635209526` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:26.800371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae716ab-7c80-4175-8448-5b2115090cf6 None admin] RBAC: Authorizing `identity:get_group(group_id=a5b3ed6cc57044e19dfae5d635209526)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:26.802004 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae716ab-7c80-4175-8448-5b2115090cf6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:26.804130 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 186/380] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jul 26 04:46:26 2022] GET /identity/v3/groups/a5b3ed6cc57044e19dfae5d635209526 => generated 226 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:26.805091 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae716ab-7c80-4175-8448-5b2115090cf6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:26.859034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd9959-4065-4096-9407-a1f3a3a870c7 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:26.898069 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719ae80-8916-41f8-9f90-bbc8f291fbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:26.898642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719ae80-8916-41f8-9f90-bbc8f291fbd0 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:26.898923 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719ae80-8916-41f8-9f90-bbc8f291fbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:26.899192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719ae80-8916-41f8-9f90-bbc8f291fbd0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:26.911193 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a719ae80-8916-41f8-9f90-bbc8f291fbd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:26.916649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-a719ae80-8916-41f8-9f90-bbc8f291fbd0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 04:46:26.916649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:26.916649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:26.916649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:26.916649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:26.916649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:26.916649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:26.916649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:26.916649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:26.916649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:26.916649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:26.916649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:26.916649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:26.916649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:26.916649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:26.916649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:26.917554 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:26.917554 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:26.917554 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:26.917554 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:26.917554 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:26.917554 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:26.917554 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:26.917554 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:26.917554 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:26.917554 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:26.917554 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:26.917554 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:26.917554 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:26.917554 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:26.917554 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:26.917554 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:26.918336 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:26.918336 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:26.918336 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:26.918336 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:26.918336 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:26.918336 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:26.918336 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:26.918336 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:26.918336 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:26.918336 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:26.918336 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 04:46:26.918336 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:26.918336 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 195/381] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jul 26 04:46:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:26.918979 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719ae80-8916-41f8-9f90-bbc8f291fbd0 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:26.939041 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae716ab-7c80-4175-8448-5b2115090cf6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:26.979328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625029d7-6f19-4e32-8dfa-526702b0cb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:26.980130 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625029d7-6f19-4e32-8dfa-526702b0cb34 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:26.980410 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625029d7-6f19-4e32-8dfa-526702b0cb34 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:26.980730 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625029d7-6f19-4e32-8dfa-526702b0cb34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:26.981538 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625029d7-6f19-4e32-8dfa-526702b0cb34 None admin] RBAC: Adding query filter params (group.id=d28be20a3a1441e59d04b97496814a5a, role.id=72bf8bd23d2c4b76ad32b7b35d6db412, scope.project.id=d12823499382429e9825c57ea3292c56) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:26.981988 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625029d7-6f19-4e32-8dfa-526702b0cb34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:26.983564 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625029d7-6f19-4e32-8dfa-526702b0cb34 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:26.990417 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 187/382] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Jul 26 04:46:26 2022] GET /identity/v3/role_assignments?group.id=d28be20a3a1441e59d04b97496814a5a&scope.project.id=d12823499382429e9825c57ea3292c56&role.id=72bf8bd23d2c4b76ad32b7b35d6db412 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:26.991264 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625029d7-6f19-4e32-8dfa-526702b0cb34 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:27.000012 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:27.030860 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719ae80-8916-41f8-9f90-bbc8f291fbd0 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:27.065420 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:27.071252 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab87732-e86c-4eb5-9de6-68ddac698bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:27.071927 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab87732-e86c-4eb5-9de6-68ddac698bcb None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:27.072206 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab87732-e86c-4eb5-9de6-68ddac698bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:27.072473 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab87732-e86c-4eb5-9de6-68ddac698bcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:27.074224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab87732-e86c-4eb5-9de6-68ddac698bcb None admin] RBAC: Adding query filter params (group.id=a5b3ed6cc57044e19dfae5d635209526, role.id=9223a9bb40484faca3a7bcfeb41a2215, scope.project.id=fce7a22bc8ff438d90035074a2c73590) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:27.074535 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab87732-e86c-4eb5-9de6-68ddac698bcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:27.076205 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab87732-e86c-4eb5-9de6-68ddac698bcb None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:27.082875 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 196/383] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Jul 26 04:46:26 2022] GET /identity/v3/role_assignments?group.id=a5b3ed6cc57044e19dfae5d635209526&scope.project.id=fce7a22bc8ff438d90035074a2c73590&role.id=9223a9bb40484faca3a7bcfeb41a2215 => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:27.089399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f7622-4929-460f-b338-861907bf8d64 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:27.089724 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f7622-4929-460f-b338-861907bf8d64 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:27.089953 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f7622-4929-460f-b338-861907bf8d64 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:27.092045 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 197/384] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:27.098448 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74634a-cac4-4e15-abb8-80fa8ca1de2f None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:27.098737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74634a-cac4-4e15-abb8-80fa8ca1de2f None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:27.099036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74634a-cac4-4e15-abb8-80fa8ca1de2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:27.120774 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625029d7-6f19-4e32-8dfa-526702b0cb34 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:27.132948 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-5e74634a-cac4-4e15-abb8-80fa8ca1de2f None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:27.134231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-5e74634a-cac4-4e15-abb8-80fa8ca1de2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:27.163156 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c11732-9a71-4c14-818a-c15e6e374f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:27.164269 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c11732-9a71-4c14-818a-c15e6e374f43 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:27.164840 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c11732-9a71-4c14-818a-c15e6e374f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:27.165373 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c11732-9a71-4c14-818a-c15e6e374f43 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:27.166417 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c11732-9a71-4c14-818a-c15e6e374f43 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:27.166967 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c11732-9a71-4c14-818a-c15e6e374f43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:27.168720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c11732-9a71-4c14-818a-c15e6e374f43 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:27.179403 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 188/385] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:46:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:27.186718 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c11732-9a71-4c14-818a-c15e6e374f43 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:27.270252 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 198/386] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:27.312779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74634a-cac4-4e15-abb8-80fa8ca1de2f None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:27.352821 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c11732-9a71-4c14-818a-c15e6e374f43 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:27.382285 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:27.406787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5ed51-bc43-4b54-b0c7-b1124e1fc196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:27.407480 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5ed51-bc43-4b54-b0c7-b1124e1fc196 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:27.407800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5ed51-bc43-4b54-b0c7-b1124e1fc196 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:27.408084 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5ed51-bc43-4b54-b0c7-b1124e1fc196 None admin] PATH_INFO: `/v3/groups/a5b3ed6cc57044e19dfae5d635209526` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:27.412618 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5ed51-bc43-4b54-b0c7-b1124e1fc196 None admin] RBAC: Authorizing `identity:get_group(group_id=a5b3ed6cc57044e19dfae5d635209526)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:27.414289 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5ed51-bc43-4b54-b0c7-b1124e1fc196 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:27.416481 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 189/387] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jul 26 04:46:27 2022] GET /identity/v3/groups/a5b3ed6cc57044e19dfae5d635209526 => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:27.424185 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5ed51-bc43-4b54-b0c7-b1124e1fc196 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:27.427962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:27.479178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74634a-cac4-4e15-abb8-80fa8ca1de2f None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:27.517500 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b117ac2-a19f-44a8-a141-4a0689059377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:27.518163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b117ac2-a19f-44a8-a141-4a0689059377 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:27.518447 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b117ac2-a19f-44a8-a141-4a0689059377 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:27.518719 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b117ac2-a19f-44a8-a141-4a0689059377 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:27.529961 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b117ac2-a19f-44a8-a141-4a0689059377 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:27.535043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-5b117ac2-a19f-44a8-a141-4a0689059377 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:46:27.535043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:27.535043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:27.535043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:27.535043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:27.535043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:27.535043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:27.535043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:27.535043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:27.535043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:27.535043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:27.535043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:27.535043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:27.535043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:27.535043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:27.535043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:27.535978 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:27.535978 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:27.535978 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:27.535978 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:27.535978 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:27.535978 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:27.535978 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:27.535978 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:27.535978 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:27.535978 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:27.535978 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:27.535978 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:27.535978 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:27.535978 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:27.535978 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:27.535978 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:27.536842 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:27.536842 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:27.536842 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:27.536842 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:27.536842 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:27.536842 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:27.536842 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:27.536842 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:27.536842 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:27.536842 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:27.536842 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:46:27.536842 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:27.536842 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 199/388] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:27.548997 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b117ac2-a19f-44a8-a141-4a0689059377 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:27.647540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5ed51-bc43-4b54-b0c7-b1124e1fc196 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:27.665068 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b117ac2-a19f-44a8-a141-4a0689059377 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:27.705333 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202274f-260e-4f5a-a7a7-19d41fb0603e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:27.706020 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202274f-260e-4f5a-a7a7-19d41fb0603e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:27.706303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202274f-260e-4f5a-a7a7-19d41fb0603e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:27.706579 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202274f-260e-4f5a-a7a7-19d41fb0603e None admin] PATH_INFO: `/v3/projects/8be3583b9359412ba4129e556beef77c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:27.708934 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a427e-f298-4944-9363-eb4655a78b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:27.709569 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a427e-f298-4944-9363-eb4655a78b90 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:27.709858 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a427e-f298-4944-9363-eb4655a78b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:27.710153 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a427e-f298-4944-9363-eb4655a78b90 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:27.710872 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5a427e-f298-4944-9363-eb4655a78b90 None admin] RBAC: Adding query filter params (name=member) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:27.711181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5a427e-f298-4944-9363-eb4655a78b90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:27.712383 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c202274f-260e-4f5a-a7a7-19d41fb0603e None admin] RBAC: Authorizing `identity:get_project(project_id=8be3583b9359412ba4129e556beef77c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:27.712652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5a427e-f298-4944-9363-eb4655a78b90 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:27.713847 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c202274f-260e-4f5a-a7a7-19d41fb0603e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:27.714442 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:27.716007 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 190/389] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:27 2022] GET /identity/v3/projects/8be3583b9359412ba4129e556beef77c => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:27.722144 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202274f-260e-4f5a-a7a7-19d41fb0603e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:27.722816 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 200/390] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:46:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:27.729662 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a427e-f298-4944-9363-eb4655a78b90 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:27.839260 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:27.874710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a427e-f298-4944-9363-eb4655a78b90 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:27.916633 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3a1c3-9b80-4389-b236-02ff0f334b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:27.917211 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3a1c3-9b80-4389-b236-02ff0f334b9b None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:27.917492 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3a1c3-9b80-4389-b236-02ff0f334b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:27.917853 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3a1c3-9b80-4389-b236-02ff0f334b9b None admin] PATH_INFO: `/v3/groups/a5b3ed6cc57044e19dfae5d635209526` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:27.918586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202274f-260e-4f5a-a7a7-19d41fb0603e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:27.919158 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb3a1c3-9b80-4389-b236-02ff0f334b9b None admin] RBAC: Authorizing `identity:get_group(group_id=a5b3ed6cc57044e19dfae5d635209526)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:27.921149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb3a1c3-9b80-4389-b236-02ff0f334b9b None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:27.927658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 201/391] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jul 26 04:46:27 2022] GET /identity/v3/groups/a5b3ed6cc57044e19dfae5d635209526 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:27.929643 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3a1c3-9b80-4389-b236-02ff0f334b9b None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:27.988470 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef24f37-5f37-43f3-861f-329597b7c8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:27.989534 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef24f37-5f37-43f3-861f-329597b7c8c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:27.990143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef24f37-5f37-43f3-861f-329597b7c8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:27.990426 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef24f37-5f37-43f3-861f-329597b7c8c1 None admin] PATH_INFO: `/v3/projects/8be3583b9359412ba4129e556beef77c/groups/a5b3ed6cc57044e19dfae5d635209526/roles/9223a9bb40484faca3a7bcfeb41a2215` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:27.995577 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef24f37-5f37-43f3-861f-329597b7c8c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=8be3583b9359412ba4129e556beef77c, group_id=a5b3ed6cc57044e19dfae5d635209526, role_id=9223a9bb40484faca3a7bcfeb41a2215)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:27.995577 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef24f37-5f37-43f3-861f-329597b7c8c1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:28.003205 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:28.025301 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 191/392] 10.222.0.17 () {58 vars in 1442 bytes} [Tue Jul 26 04:46:27 2022] PUT /identity/v3/projects/8be3583b9359412ba4129e556beef77c/groups/a5b3ed6cc57044e19dfae5d635209526/roles/9223a9bb40484faca3a7bcfeb41a2215 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:46:28.059605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3a1c3-9b80-4389-b236-02ff0f334b9b None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:28.106014 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a5d784-88ce-497d-b2b9-4848cd847fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:28.106677 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5d784-88ce-497d-b2b9-4848cd847fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:28.107053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5d784-88ce-497d-b2b9-4848cd847fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:28.107330 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5d784-88ce-497d-b2b9-4848cd847fb4 None admin] PATH_INFO: `/v3/projects/8be3583b9359412ba4129e556beef77c` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:28.112633 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a5d784-88ce-497d-b2b9-4848cd847fb4 None admin] RBAC: Authorizing `identity:get_project(project_id=8be3583b9359412ba4129e556beef77c)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:28.114181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a5d784-88ce-497d-b2b9-4848cd847fb4 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:28.116313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 202/393] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:27 2022] GET /identity/v3/projects/8be3583b9359412ba4129e556beef77c => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:28.122154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef24f37-5f37-43f3-861f-329597b7c8c1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:28.285179 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef24f37-5f37-43f3-861f-329597b7c8c1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:28.315084 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:28.342530 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf057e4-7c7d-4091-af1d-e3d9c7253f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:28.342530 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf057e4-7c7d-4091-af1d-e3d9c7253f19 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:28.342530 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf057e4-7c7d-4091-af1d-e3d9c7253f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:28.342530 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf057e4-7c7d-4091-af1d-e3d9c7253f19 None admin] PATH_INFO: `/v3/projects/8be3583b9359412ba4129e556beef77c/groups/a5b3ed6cc57044e19dfae5d635209526/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:28.344895 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf057e4-7c7d-4091-af1d-e3d9c7253f19 None admin] RBAC: Authorizing `identity:create_grant(project_id=8be3583b9359412ba4129e556beef77c, group_id=a5b3ed6cc57044e19dfae5d635209526, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:28.346451 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf057e4-7c7d-4091-af1d-e3d9c7253f19 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:28.364951 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 192/394] 10.222.0.17 () {58 vars in 1442 bytes} [Tue Jul 26 04:46:28 2022] PUT /identity/v3/projects/8be3583b9359412ba4129e556beef77c/groups/a5b3ed6cc57044e19dfae5d635209526/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:46:28.783661 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120aade-a87d-4dcc-9d9e-4f3b1a0b9fe7 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:28.784230 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120aade-a87d-4dcc-9d9e-4f3b1a0b9fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:28.784507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120aade-a87d-4dcc-9d9e-4f3b1a0b9fe7 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:28.786300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 203/395] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:28.794583 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511d68f-e750-4af3-8cb1-1f694da782e0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:28.795051 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511d68f-e750-4af3-8cb1-1f694da782e0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:28.795461 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511d68f-e750-4af3-8cb1-1f694da782e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:28.821888 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5511d68f-e750-4af3-8cb1-1f694da782e0 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:28.823111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5511d68f-e750-4af3-8cb1-1f694da782e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:28.930063 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 193/396] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 04:46:28.973205 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120aade-a87d-4dcc-9d9e-4f3b1a0b9fe7 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:29.097612 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120aade-a87d-4dcc-9d9e-4f3b1a0b9fe7 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:29.133926 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:29.135277 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c53421-b79c-44e2-83d2-6885b91ea3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:29.136043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c53421-b79c-44e2-83d2-6885b91ea3d4 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:29.136401 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c53421-b79c-44e2-83d2-6885b91ea3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:29.136720 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c53421-b79c-44e2-83d2-6885b91ea3d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:29.148289 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c53421-b79c-44e2-83d2-6885b91ea3d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:29.153343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-d2c53421-b79c-44e2-83d2-6885b91ea3d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:46:29.153343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:29.153343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:29.153343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:29.153343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:29.153343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:29.153343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:29.153343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:29.153343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:29.153343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:29.153343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:29.153343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:29.153343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:29.153343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:29.153343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:29.153343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:29.154294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:29.154294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:29.154294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:29.154294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:29.154294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:29.154294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:29.154294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:29.154294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:29.154294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:29.154294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:29.154294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:29.154294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:29.154294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:29.154294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:29.154294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:29.154294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:29.155073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:29.155073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:29.155073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:29.155073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:29.155073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:29.155073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:29.155073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:29.155073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:29.155073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:29.155073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:29.155073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:46:29.155073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:29.155073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 204/397] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:29.160645 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511d68f-e750-4af3-8cb1-1f694da782e0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:29.324350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511d68f-e750-4af3-8cb1-1f694da782e0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:29.345476 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:29.378233 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f7779-19d0-4fea-bb0b-68c00b6d4f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:29.379676 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f7779-19d0-4fea-bb0b-68c00b6d4f22 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:29.380187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f7779-19d0-4fea-bb0b-68c00b6d4f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:29.380698 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f7779-19d0-4fea-bb0b-68c00b6d4f22 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:29.381608 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f7779-19d0-4fea-bb0b-68c00b6d4f22 None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:29.382116 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f7779-19d0-4fea-bb0b-68c00b6d4f22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:29.383722 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f7779-19d0-4fea-bb0b-68c00b6d4f22 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:29.397090 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 194/398] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:46:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:29.402904 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c4b65-305b-40b8-947c-85edd33beba0 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:29.403221 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c4b65-305b-40b8-947c-85edd33beba0 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:29.403441 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f7779-19d0-4fea-bb0b-68c00b6d4f22 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:29.403538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c4b65-305b-40b8-947c-85edd33beba0 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:29.405443 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 205/399] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:29.412393 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3d9bc-50c7-417a-baa6-62c534b18492 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:29.412694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3d9bc-50c7-417a-baa6-62c534b18492 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:29.412944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3d9bc-50c7-417a-baa6-62c534b18492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:29.450650 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-42c3d9bc-50c7-417a-baa6-62c534b18492 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:29.453120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-42c3d9bc-50c7-417a-baa6-62c534b18492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:29.556430 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 206/400] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 04:46:29.570059 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f7779-19d0-4fea-bb0b-68c00b6d4f22 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:29.586639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3d9bc-50c7-417a-baa6-62c534b18492 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:29.615329 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e70ec6-bcf6-45ab-ab48-460549c83671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:29.616021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e70ec6-bcf6-45ab-ab48-460549c83671 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:29.616303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e70ec6-bcf6-45ab-ab48-460549c83671 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:29.616581 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e70ec6-bcf6-45ab-ab48-460549c83671 None admin] PATH_INFO: `/v3/groups/a5b3ed6cc57044e19dfae5d635209526` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:29.617857 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e70ec6-bcf6-45ab-ab48-460549c83671 None admin] RBAC: Authorizing `identity:get_group(group_id=a5b3ed6cc57044e19dfae5d635209526)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:29.619306 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e70ec6-bcf6-45ab-ab48-460549c83671 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:29.621253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 195/401] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jul 26 04:46:29 2022] GET /identity/v3/groups/a5b3ed6cc57044e19dfae5d635209526 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:29.629867 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e70ec6-bcf6-45ab-ab48-460549c83671 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:29.719218 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:29.808372 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3d9bc-50c7-417a-baa6-62c534b18492 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:29.834647 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:29.847912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e70ec6-bcf6-45ab-ab48-460549c83671 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:29.870615 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52bedd-1af0-41f1-87bf-2734bf52e5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:29.871297 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52bedd-1af0-41f1-87bf-2734bf52e5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:29.871596 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52bedd-1af0-41f1-87bf-2734bf52e5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:29.871871 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52bedd-1af0-41f1-87bf-2734bf52e5d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:29.896571 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52bedd-1af0-41f1-87bf-2734bf52e5d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:29.901685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-7a52bedd-1af0-41f1-87bf-2734bf52e5d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:46:29.901685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:29.901685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:29.901685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:29.901685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:29.901685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:29.901685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:29.901685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:29.901685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:29.901685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:29.901685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:29.901685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:29.901685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:29.901685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:29.901685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:29.901685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:29.902605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:29.902605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:29.902605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:29.902605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:29.902605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:29.902605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:29.902605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:29.902605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:29.902605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:29.902605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:29.902605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:29.902605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:29.902605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:29.902605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:29.902605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:29.902605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:29.903381 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:29.903381 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:29.903381 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:29.903381 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:29.903381 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:29.903381 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:29.903381 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:29.903381 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:29.903381 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:29.903381 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:29.903381 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:46:29.903381 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:29.903381 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 207/402] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:46:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:46:29.912803 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52bedd-1af0-41f1-87bf-2734bf52e5d1 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:29.922225 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97ee0e-9963-44c5-90f8-a449501881fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:29.923055 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97ee0e-9963-44c5-90f8-a449501881fd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:29.923338 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97ee0e-9963-44c5-90f8-a449501881fd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:29.923815 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97ee0e-9963-44c5-90f8-a449501881fd None admin] PATH_INFO: `/v3/projects/fce7a22bc8ff438d90035074a2c73590` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:29.928408 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f97ee0e-9963-44c5-90f8-a449501881fd None admin] RBAC: Authorizing `identity:get_project(project_id=fce7a22bc8ff438d90035074a2c73590)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:29.930221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f97ee0e-9963-44c5-90f8-a449501881fd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:29.936772 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 196/403] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:29 2022] GET /identity/v3/projects/fce7a22bc8ff438d90035074a2c73590 => generated 302 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:29.943610 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97ee0e-9963-44c5-90f8-a449501881fd None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:30.089206 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52bedd-1af0-41f1-87bf-2734bf52e5d1 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:30.123900 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97ee0e-9963-44c5-90f8-a449501881fd None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:30.136817 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:30.154916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b3686-e212-4340-8898-42bd7bd53677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:30.155518 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b3686-e212-4340-8898-42bd7bd53677 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:30.155939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b3686-e212-4340-8898-42bd7bd53677 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:30.156271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b3686-e212-4340-8898-42bd7bd53677 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:30.157303 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b3686-e212-4340-8898-42bd7bd53677 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:30.157618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b3686-e212-4340-8898-42bd7bd53677 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:30.159839 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b3686-e212-4340-8898-42bd7bd53677 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:30.173723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3399a3-f928-445d-b3e7-67ed4822c596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:30.175007 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3399a3-f928-445d-b3e7-67ed4822c596 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:30.175462 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3399a3-f928-445d-b3e7-67ed4822c596 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:30.175933 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3399a3-f928-445d-b3e7-67ed4822c596 None admin] PATH_INFO: `/v3/projects/fce7a22bc8ff438d90035074a2c73590/groups/a5b3ed6cc57044e19dfae5d635209526/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:30.179878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 208/404] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:30.180017 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3399a3-f928-445d-b3e7-67ed4822c596 None admin] RBAC: Authorizing `identity:create_grant(project_id=fce7a22bc8ff438d90035074a2c73590, group_id=a5b3ed6cc57044e19dfae5d635209526, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:30.181422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea6546-7a37-4399-af25-b5c5eb3b1757 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:30.181813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea6546-7a37-4399-af25-b5c5eb3b1757 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:30.182185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea6546-7a37-4399-af25-b5c5eb3b1757 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:30.182433 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3399a3-f928-445d-b3e7-67ed4822c596 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:30.188198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 209/405] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:30.188198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea6546-7a37-4399-af25-b5c5eb3b1757 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:30.207274 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 197/406] 10.222.0.17 () {58 vars in 1442 bytes} [Tue Jul 26 04:46:29 2022] PUT /identity/v3/projects/fce7a22bc8ff438d90035074a2c73590/groups/a5b3ed6cc57044e19dfae5d635209526/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 04:46:30.209231 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4cc24-84d8-4b27-88e2-557cd60e075f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:30.209733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4cc24-84d8-4b27-88e2-557cd60e075f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:30.210216 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4cc24-84d8-4b27-88e2-557cd60e075f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:30.234906 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-bbf4cc24-84d8-4b27-88e2-557cd60e075f None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:30.236559 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bbf4cc24-84d8-4b27-88e2-557cd60e075f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:30.260883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:30.328710 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 198/407] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:30.355128 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4cc24-84d8-4b27-88e2-557cd60e075f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:30.404394 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea6546-7a37-4399-af25-b5c5eb3b1757 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:30.461407 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd6a46-2b2f-4d1b-a2d2-167ef6b67a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:30.463365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd6a46-2b2f-4d1b-a2d2-167ef6b67a9e None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:30.463737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd6a46-2b2f-4d1b-a2d2-167ef6b67a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:30.464099 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd6a46-2b2f-4d1b-a2d2-167ef6b67a9e None admin] PATH_INFO: `/v3/groups/d28be20a3a1441e59d04b97496814a5a` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:30.465830 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd6a46-2b2f-4d1b-a2d2-167ef6b67a9e None admin] RBAC: Authorizing `identity:get_group(group_id=d28be20a3a1441e59d04b97496814a5a)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:30.467098 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd6a46-2b2f-4d1b-a2d2-167ef6b67a9e None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:30.469550 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 210/408] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jul 26 04:46:30 2022] GET /identity/v3/groups/d28be20a3a1441e59d04b97496814a5a => generated 229 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:30.474739 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd6a46-2b2f-4d1b-a2d2-167ef6b67a9e None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:30.479940 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4cc24-84d8-4b27-88e2-557cd60e075f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:30.483897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:30.520872 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc97f4-80ac-41d5-a420-c02fd031a95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:30.521903 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc97f4-80ac-41d5-a420-c02fd031a95c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:30.522391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc97f4-80ac-41d5-a420-c02fd031a95c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:30.522829 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc97f4-80ac-41d5-a420-c02fd031a95c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:30.535741 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc97f4-80ac-41d5-a420-c02fd031a95c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:30.542284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-5abc97f4-80ac-41d5-a420-c02fd031a95c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 04:46:30.542284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:30.542284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:30.542284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:30.542284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:30.542284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:30.542284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:30.542284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:30.542284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:30.542284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:30.542284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:30.542284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:30.542284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:30.542284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:30.542284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:30.542284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:30.543345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:30.543345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:30.543345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:30.543345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:30.543345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:30.543345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:30.543345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:30.543345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:30.543345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:30.543345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:30.543345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:30.543345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:30.543345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:30.543345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:30.543345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:30.543345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:30.544200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:30.544200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:30.544200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:30.544200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:30.544200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:30.544200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:30.544200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:30.544200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:30.544200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:30.544200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:30.544200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 04:46:30.544200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:30.551774 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 199/409] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jul 26 04:46:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:46:30.553440 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc97f4-80ac-41d5-a420-c02fd031a95c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:30.588515 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd6a46-2b2f-4d1b-a2d2-167ef6b67a9e None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:30.596880 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:30.657295 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b129e6-d6cc-4f91-85b4-8b5dcc6c1f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:30.657994 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b129e6-d6cc-4f91-85b4-8b5dcc6c1f8a None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:30.658271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b129e6-d6cc-4f91-85b4-8b5dcc6c1f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:30.658541 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b129e6-d6cc-4f91-85b4-8b5dcc6c1f8a None admin] PATH_INFO: `/v3/projects/d12823499382429e9825c57ea3292c56` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:30.663186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b129e6-d6cc-4f91-85b4-8b5dcc6c1f8a None admin] RBAC: Authorizing `identity:get_project(project_id=d12823499382429e9825c57ea3292c56)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:30.664773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b129e6-d6cc-4f91-85b4-8b5dcc6c1f8a None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:30.666259 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc97f4-80ac-41d5-a420-c02fd031a95c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:30.666803 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 211/410] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:30 2022] GET /identity/v3/projects/d12823499382429e9825c57ea3292c56 => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:30.672305 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b129e6-d6cc-4f91-85b4-8b5dcc6c1f8a None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:30.704518 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b2801-09cc-4129-b3f7-a4d155ef15b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:30.705550 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b2801-09cc-4129-b3f7-a4d155ef15b3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:30.706043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b2801-09cc-4129-b3f7-a4d155ef15b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:30.706524 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b2801-09cc-4129-b3f7-a4d155ef15b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:30.707447 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b2801-09cc-4129-b3f7-a4d155ef15b3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:30.708078 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b2801-09cc-4129-b3f7-a4d155ef15b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:30.709782 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b2801-09cc-4129-b3f7-a4d155ef15b3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:30.723753 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 200/411] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:46:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:30.725805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b2801-09cc-4129-b3f7-a4d155ef15b3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:30.763956 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:30.833024 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b2801-09cc-4129-b3f7-a4d155ef15b3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:30.874174 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa35b8c-3227-4d35-af35-0d0e1747b2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:30.874843 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa35b8c-3227-4d35-af35-0d0e1747b2ea None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:30.875122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa35b8c-3227-4d35-af35-0d0e1747b2ea None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:30.875398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa35b8c-3227-4d35-af35-0d0e1747b2ea None admin] PATH_INFO: `/v3/groups/a5b3ed6cc57044e19dfae5d635209526` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:30.877333 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa35b8c-3227-4d35-af35-0d0e1747b2ea None admin] RBAC: Authorizing `identity:get_group(group_id=a5b3ed6cc57044e19dfae5d635209526)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:30.878788 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa35b8c-3227-4d35-af35-0d0e1747b2ea None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:30.882624 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b129e6-d6cc-4f91-85b4-8b5dcc6c1f8a None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:30.885048 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 201/412] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jul 26 04:46:30 2022] GET /identity/v3/groups/a5b3ed6cc57044e19dfae5d635209526 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:30.886464 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ae610-2967-49c3-818b-43e8df076701 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:30.886764 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ae610-2967-49c3-818b-43e8df076701 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:30.887067 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ae610-2967-49c3-818b-43e8df076701 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:30.889253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 202/413] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:30.890108 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ae610-2967-49c3-818b-43e8df076701 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:30.923668 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c40d3-e51f-4ecd-8e13-30f083012f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:30.924792 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c40d3-e51f-4ecd-8e13-30f083012f44 None admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:30.925302 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c40d3-e51f-4ecd-8e13-30f083012f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:30.925770 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c40d3-e51f-4ecd-8e13-30f083012f44 None admin] PATH_INFO: `/v3/projects/d12823499382429e9825c57ea3292c56/groups/d28be20a3a1441e59d04b97496814a5a/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:30.927984 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c40d3-e51f-4ecd-8e13-30f083012f44 None admin] RBAC: Authorizing `identity:create_grant(project_id=d12823499382429e9825c57ea3292c56, group_id=d28be20a3a1441e59d04b97496814a5a, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:30.929745 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c40d3-e51f-4ecd-8e13-30f083012f44 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:30.948176 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 212/414] 10.222.0.17 () {58 vars in 1442 bytes} [Tue Jul 26 04:46:30 2022] PUT /identity/v3/projects/d12823499382429e9825c57ea3292c56/groups/d28be20a3a1441e59d04b97496814a5a/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 04:46:30.949784 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14504929-66f8-47c2-894f-30819bcd5361 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:30.950217 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14504929-66f8-47c2-894f-30819bcd5361 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:30.950636 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14504929-66f8-47c2-894f-30819bcd5361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:30.976983 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-14504929-66f8-47c2-894f-30819bcd5361 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:30.978200 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-14504929-66f8-47c2-894f-30819bcd5361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:31.035586 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:31.056001 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:31.064231 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ae610-2967-49c3-818b-43e8df076701 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:31.095232 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 213/415] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:31.097287 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bbcd3d-2ed9-4c3a-8066-3457d3908d8e None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:31.097783 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bbcd3d-2ed9-4c3a-8066-3457d3908d8e None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:31.098247 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bbcd3d-2ed9-4c3a-8066-3457d3908d8e None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:31.101071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 214/416] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:31.107126 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec34561-8388-47a8-85b2-72b3e573bf09 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:31.107712 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec34561-8388-47a8-85b2-72b3e573bf09 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:31.108176 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec34561-8388-47a8-85b2-72b3e573bf09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:31.112201 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc11c1-7963-477d-930b-4f82b0f719fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:31.112972 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc11c1-7963-477d-930b-4f82b0f719fe None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:31.113308 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc11c1-7963-477d-930b-4f82b0f719fe None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:31.113594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc11c1-7963-477d-930b-4f82b0f719fe None admin] PATH_INFO: `/v3/projects/fce7a22bc8ff438d90035074a2c73590` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:31.114907 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc11c1-7963-477d-930b-4f82b0f719fe None admin] RBAC: Authorizing `identity:get_project(project_id=fce7a22bc8ff438d90035074a2c73590)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:31.116583 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc11c1-7963-477d-930b-4f82b0f719fe None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:31.118451 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 203/417] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:30 2022] GET /identity/v3/projects/fce7a22bc8ff438d90035074a2c73590 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:31.124758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc11c1-7963-477d-930b-4f82b0f719fe None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:31.130605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-dec34561-8388-47a8-85b2-72b3e573bf09 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:31.132209 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-dec34561-8388-47a8-85b2-72b3e573bf09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:31.219539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 215/418] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:31.220447 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec34561-8388-47a8-85b2-72b3e573bf09 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:31.261761 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc11c1-7963-477d-930b-4f82b0f719fe None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:31.299106 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5848a3fc-462e-4118-a7d4-e780e4eafa4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:31.299973 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848a3fc-462e-4118-a7d4-e780e4eafa4b None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:31.300264 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848a3fc-462e-4118-a7d4-e780e4eafa4b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:31.300540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848a3fc-462e-4118-a7d4-e780e4eafa4b None admin] PATH_INFO: `/v3/projects/fce7a22bc8ff438d90035074a2c73590/groups/a5b3ed6cc57044e19dfae5d635209526/roles/9223a9bb40484faca3a7bcfeb41a2215` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:31.302868 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5848a3fc-462e-4118-a7d4-e780e4eafa4b None admin] RBAC: Authorizing `identity:create_grant(project_id=fce7a22bc8ff438d90035074a2c73590, group_id=a5b3ed6cc57044e19dfae5d635209526, role_id=9223a9bb40484faca3a7bcfeb41a2215)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:31.304499 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5848a3fc-462e-4118-a7d4-e780e4eafa4b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:31.325170 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 204/419] 10.222.0.17 () {58 vars in 1442 bytes} [Tue Jul 26 04:46:31 2022] PUT /identity/v3/projects/fce7a22bc8ff438d90035074a2c73590/groups/a5b3ed6cc57044e19dfae5d635209526/roles/9223a9bb40484faca3a7bcfeb41a2215 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:46:31.326275 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5848a3fc-462e-4118-a7d4-e780e4eafa4b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:31.336702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:31.352746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:31.356836 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec34561-8388-47a8-85b2-72b3e573bf09 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:31.458039 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5848a3fc-462e-4118-a7d4-e780e4eafa4b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:31.482163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2d738-c510-41a7-98a6-6572b7e114f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:31.483280 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2d738-c510-41a7-98a6-6572b7e114f1 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:31.483823 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2d738-c510-41a7-98a6-6572b7e114f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:31.484285 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2d738-c510-41a7-98a6-6572b7e114f1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:31.498901 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b70b23-a0b5-4e7b-bccb-9a0ba4b29349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:31.499607 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b70b23-a0b5-4e7b-bccb-9a0ba4b29349 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:31.499892 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b70b23-a0b5-4e7b-bccb-9a0ba4b29349 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:31.500227 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b70b23-a0b5-4e7b-bccb-9a0ba4b29349 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:31.515623 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa2d738-c510-41a7-98a6-6572b7e114f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:31.521557 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-caa2d738-c510-41a7-98a6-6572b7e114f1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 04:46:31.521557 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:31.521557 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:31.521557 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:31.521557 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:31.521557 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:31.521557 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:31.521557 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:31.521557 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:31.521557 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:31.521557 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:31.521557 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:31.521557 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:31.521557 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:31.521557 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:31.521557 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:31.522507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:31.522507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:31.522507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:31.522507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:31.522507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:31.522507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:31.522507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:31.522507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:31.522507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:31.522507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:31.522507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:31.522507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:31.522507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:31.522507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:31.522507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:31.522507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:31.523544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:31.523544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:31.523544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:31.523544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:31.523544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:31.523544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:31.523544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:31.523544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:31.523544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:31.523544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:31.523544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 04:46:31.523544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:31.523544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 216/420] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jul 26 04:46:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:31.529014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b70b23-a0b5-4e7b-bccb-9a0ba4b29349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:31.533431 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2d738-c510-41a7-98a6-6572b7e114f1 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:31.534355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a2b70b23-a0b5-4e7b-bccb-9a0ba4b29349 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:46:31.534355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:31.534355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:31.534355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:31.534355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:31.534355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:31.534355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:31.534355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:31.534355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:31.534355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:31.534355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:31.534355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:31.534355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:31.534355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:31.534355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:31.534355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:31.535230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:31.535230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:31.535230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:31.535230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:31.535230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:31.535230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:31.535230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:31.535230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:31.535230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:31.535230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:31.535230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:31.535230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:31.535230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:31.535230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:31.535230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:31.535230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:31.536468 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:31.536468 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:31.536468 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:31.536468 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:31.536468 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:31.536468 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:31.536468 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:31.536468 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:31.536468 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:31.536468 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:31.536468 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:46:31.536468 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:31.536468 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 205/421] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:31.543746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b70b23-a0b5-4e7b-bccb-9a0ba4b29349 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:31.633530 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:31.661847 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b70b23-a0b5-4e7b-bccb-9a0ba4b29349 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:31.674384 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2d738-c510-41a7-98a6-6572b7e114f1 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:31.722067 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f6fe7-c477-4c7c-8c2f-ed13da0e296b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:31.722715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f6fe7-c477-4c7c-8c2f-ed13da0e296b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:31.722997 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f6fe7-c477-4c7c-8c2f-ed13da0e296b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:31.723270 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f6fe7-c477-4c7c-8c2f-ed13da0e296b None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:31.724032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f6fe7-c477-4c7c-8c2f-ed13da0e296b None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:31.724335 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f6fe7-c477-4c7c-8c2f-ed13da0e296b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:31.725740 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f6fe7-c477-4c7c-8c2f-ed13da0e296b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:31.743428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 206/422] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:46:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:31.745898 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d2670-db86-4581-9b90-69784a0d594a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:31.746568 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d2670-db86-4581-9b90-69784a0d594a None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:31.746849 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d2670-db86-4581-9b90-69784a0d594a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:31.747122 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d2670-db86-4581-9b90-69784a0d594a None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:31.748010 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d2670-db86-4581-9b90-69784a0d594a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:31.748324 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d2670-db86-4581-9b90-69784a0d594a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:31.749848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d2670-db86-4581-9b90-69784a0d594a None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:31.752510 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f6fe7-c477-4c7c-8c2f-ed13da0e296b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:31.772998 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 217/423] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:46:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:31.778613 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d2670-db86-4581-9b90-69784a0d594a None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:31.806024 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:31.935301 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f6fe7-c477-4c7c-8c2f-ed13da0e296b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:31.961737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d2670-db86-4581-9b90-69784a0d594a None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:31.974315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56124084-00d6-4794-81f2-94653edc8208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:31.974951 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56124084-00d6-4794-81f2-94653edc8208 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:31.975231 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56124084-00d6-4794-81f2-94653edc8208 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:31.976210 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56124084-00d6-4794-81f2-94653edc8208 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:31.990288 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:32.010270 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0f4d5-4a05-4f94-9c2d-6ea9995b896a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:32.010907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0f4d5-4a05-4f94-9c2d-6ea9995b896a None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:32.011193 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0f4d5-4a05-4f94-9c2d-6ea9995b896a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:32.011649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0f4d5-4a05-4f94-9c2d-6ea9995b896a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:32.079273 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56124084-00d6-4794-81f2-94653edc8208 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:32.080850 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56124084-00d6-4794-81f2-94653edc8208 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:32.187466 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf0f4d5-4a05-4f94-9c2d-6ea9995b896a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:32.189032 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf0f4d5-4a05-4f94-9c2d-6ea9995b896a None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:32.198778 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 207/424] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:46:32.210869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56124084-00d6-4794-81f2-94653edc8208 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:32.217420 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:32.293660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:32.397830 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56124084-00d6-4794-81f2-94653edc8208 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:32.410943 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 218/425] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 633 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 04:46:32.416177 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0f4d5-4a05-4f94-9c2d-6ea9995b896a None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:32.437773 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b0ae8-5b92-45ee-8e02-1fe5513d6afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:32.438439 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b0ae8-5b92-45ee-8e02-1fe5513d6afc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:32.438721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b0ae8-5b92-45ee-8e02-1fe5513d6afc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:32.438995 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b0ae8-5b92-45ee-8e02-1fe5513d6afc None admin] PATH_INFO: `/v3/projects/8be3583b9359412ba4129e556beef77c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:32.440832 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b0ae8-5b92-45ee-8e02-1fe5513d6afc None admin] RBAC: Authorizing `identity:get_project(project_id=8be3583b9359412ba4129e556beef77c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:32.442307 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b0ae8-5b92-45ee-8e02-1fe5513d6afc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:32.444045 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 208/426] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:32 2022] GET /identity/v3/projects/8be3583b9359412ba4129e556beef77c => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:32.454081 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b0ae8-5b92-45ee-8e02-1fe5513d6afc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:32.565445 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b0ae8-5b92-45ee-8e02-1fe5513d6afc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:32.573303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:32.606988 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba2e09-3dd5-4f11-9fa1-1ccdf098514e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:32.609411 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba2e09-3dd5-4f11-9fa1-1ccdf098514e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:32.610120 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba2e09-3dd5-4f11-9fa1-1ccdf098514e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:32.611259 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba2e09-3dd5-4f11-9fa1-1ccdf098514e None admin] PATH_INFO: `/v3/groups/a5b3ed6cc57044e19dfae5d635209526` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:32.612963 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba2e09-3dd5-4f11-9fa1-1ccdf098514e None admin] RBAC: Authorizing `identity:get_group(group_id=a5b3ed6cc57044e19dfae5d635209526)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:32.615399 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba2e09-3dd5-4f11-9fa1-1ccdf098514e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:32.617448 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 209/427] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jul 26 04:46:32 2022] GET /identity/v3/groups/a5b3ed6cc57044e19dfae5d635209526 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:32.624474 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba2e09-3dd5-4f11-9fa1-1ccdf098514e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:32.643691 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0f4d5-4a05-4f94-9c2d-6ea9995b896a None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:32.656938 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:32.679083 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1da9f9-6f65-428f-9afb-17b840d5b9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:32.679785 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1da9f9-6f65-428f-9afb-17b840d5b9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:32.680077 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1da9f9-6f65-428f-9afb-17b840d5b9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:32.680417 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1da9f9-6f65-428f-9afb-17b840d5b9f0 None admin] PATH_INFO: `/v3/projects/8be3583b9359412ba4129e556beef77c` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:32.682133 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1da9f9-6f65-428f-9afb-17b840d5b9f0 None admin] RBAC: Authorizing `identity:get_project(project_id=8be3583b9359412ba4129e556beef77c)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:32.683909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1da9f9-6f65-428f-9afb-17b840d5b9f0 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:32.686863 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 219/428] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:32 2022] GET /identity/v3/projects/8be3583b9359412ba4129e556beef77c => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:32.692566 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1da9f9-6f65-428f-9afb-17b840d5b9f0 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:32.812622 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1da9f9-6f65-428f-9afb-17b840d5b9f0 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:32.837295 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba2e09-3dd5-4f11-9fa1-1ccdf098514e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:32.853192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdba140-3b8b-4486-b30a-3bd000534c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:32.854158 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdba140-3b8b-4486-b30a-3bd000534c13 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:32.854648 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdba140-3b8b-4486-b30a-3bd000534c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:32.855097 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdba140-3b8b-4486-b30a-3bd000534c13 None admin] PATH_INFO: `/v3/groups/a5b3ed6cc57044e19dfae5d635209526` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:32.856775 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdba140-3b8b-4486-b30a-3bd000534c13 None admin] RBAC: Authorizing `identity:get_group(group_id=a5b3ed6cc57044e19dfae5d635209526)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:32.858445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdba140-3b8b-4486-b30a-3bd000534c13 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:32.860450 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 220/429] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jul 26 04:46:32 2022] GET /identity/v3/groups/a5b3ed6cc57044e19dfae5d635209526 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:32.866564 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60e989-e2a2-48b1-84f8-84896fcdbf2c None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:32.867294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60e989-e2a2-48b1-84f8-84896fcdbf2c None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:32.867777 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60e989-e2a2-48b1-84f8-84896fcdbf2c None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:32.869633 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 221/430] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:32.870642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60e989-e2a2-48b1-84f8-84896fcdbf2c None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:32.882649 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fc7ba-dc85-4480-8aaf-ad7cb20451fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:32.883301 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fc7ba-dc85-4480-8aaf-ad7cb20451fb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:32.883616 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fc7ba-dc85-4480-8aaf-ad7cb20451fb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:32.883893 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fc7ba-dc85-4480-8aaf-ad7cb20451fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:32.884688 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0fc7ba-dc85-4480-8aaf-ad7cb20451fb None admin] RBAC: Adding query filter params (group.id=a5b3ed6cc57044e19dfae5d635209526, role.id=7519356859fb40508c25bec8161207fd, scope.project.id=8be3583b9359412ba4129e556beef77c) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:32.884997 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0fc7ba-dc85-4480-8aaf-ad7cb20451fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:32.886461 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0fc7ba-dc85-4480-8aaf-ad7cb20451fb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:32.899111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 210/431] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Jul 26 04:46:32 2022] GET /identity/v3/role_assignments?group.id=a5b3ed6cc57044e19dfae5d635209526&scope.project.id=8be3583b9359412ba4129e556beef77c&role.id=7519356859fb40508c25bec8161207fd => generated 611 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:32.899111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30117485-2f75-41ce-998e-f3c093723b17 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:32.899111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30117485-2f75-41ce-998e-f3c093723b17 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:32.899111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30117485-2f75-41ce-998e-f3c093723b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:32.933122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-30117485-2f75-41ce-998e-f3c093723b17 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:32.934355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-30117485-2f75-41ce-998e-f3c093723b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:32.973794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:32.986348 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:33.022929 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 211/432] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:33.046201 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30117485-2f75-41ce-998e-f3c093723b17 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:33.098647 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60e989-e2a2-48b1-84f8-84896fcdbf2c None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:33.148937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57252ee1-d96a-4b50-a132-ec8267565b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:33.149633 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57252ee1-d96a-4b50-a132-ec8267565b10 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:33.149865 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57252ee1-d96a-4b50-a132-ec8267565b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:33.150144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57252ee1-d96a-4b50-a132-ec8267565b10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:33.150960 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57252ee1-d96a-4b50-a132-ec8267565b10 None admin] RBAC: Adding query filter params (group.id=a5b3ed6cc57044e19dfae5d635209526, role.id=9223a9bb40484faca3a7bcfeb41a2215, scope.project.id=8be3583b9359412ba4129e556beef77c) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:33.151395 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57252ee1-d96a-4b50-a132-ec8267565b10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:33.153032 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57252ee1-d96a-4b50-a132-ec8267565b10 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:33.162418 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 222/433] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Jul 26 04:46:32 2022] GET /identity/v3/role_assignments?group.id=a5b3ed6cc57044e19dfae5d635209526&scope.project.id=8be3583b9359412ba4129e556beef77c&role.id=9223a9bb40484faca3a7bcfeb41a2215 => generated 611 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:33.195241 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30117485-2f75-41ce-998e-f3c093723b17 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:33.243187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7211a51-3f7c-4ad4-b4ea-3689b2aae984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:33.243916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7211a51-3f7c-4ad4-b4ea-3689b2aae984 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:33.244198 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7211a51-3f7c-4ad4-b4ea-3689b2aae984 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:33.244466 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7211a51-3f7c-4ad4-b4ea-3689b2aae984 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:33.253404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b09afa-4661-448a-b225-0c5ec3cad0f9 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:33.253828 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b09afa-4661-448a-b225-0c5ec3cad0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:33.254006 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b09afa-4661-448a-b225-0c5ec3cad0f9 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:33.255510 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 223/434] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:46:33.260019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7211a51-3f7c-4ad4-b4ea-3689b2aae984 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:33.262671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571191f-2291-48c6-837d-7f99e0311b2c None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:33.263293 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571191f-2291-48c6-837d-7f99e0311b2c None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:33.263805 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571191f-2291-48c6-837d-7f99e0311b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:33.265299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c7211a51-3f7c-4ad4-b4ea-3689b2aae984 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:46:33.265299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:33.265299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:33.265299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:33.265299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:33.265299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:33.265299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:33.265299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:33.265299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:33.265299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:33.265299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:33.265299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:33.265299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:33.265299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:33.265299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:33.265299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:33.266197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:33.266197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:33.266197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:33.266197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:33.266197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:33.266197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:33.266197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:33.266197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:33.266197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:33.266197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:33.266197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:33.266197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:33.266197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:33.266197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:33.266197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:33.266197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:33.267428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:33.267428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:33.267428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:33.267428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:33.267428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:33.267428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:33.267428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:33.267428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:33.267428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:33.267428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:33.267428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 04:46:33.267428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:33.267428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 212/435] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:46:33.273258 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7211a51-3f7c-4ad4-b4ea-3689b2aae984 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:33.290355 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-f571191f-2291-48c6-837d-7f99e0311b2c None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:33.292451 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-f571191f-2291-48c6-837d-7f99e0311b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:33.314123 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:33.387663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 224/436] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 04:46:33.410144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571191f-2291-48c6-837d-7f99e0311b2c None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:33.429748 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7211a51-3f7c-4ad4-b4ea-3689b2aae984 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:33.483065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:33.505016 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bac8f8-fbbd-4cc2-a862-d8b4a0ad865c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:33.505649 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bac8f8-fbbd-4cc2-a862-d8b4a0ad865c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:33.505933 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bac8f8-fbbd-4cc2-a862-d8b4a0ad865c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:33.506206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bac8f8-fbbd-4cc2-a862-d8b4a0ad865c None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:33.506967 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bac8f8-fbbd-4cc2-a862-d8b4a0ad865c None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:33.507248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bac8f8-fbbd-4cc2-a862-d8b4a0ad865c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:33.508705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bac8f8-fbbd-4cc2-a862-d8b4a0ad865c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:33.522404 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 213/437] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:46:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:33.528030 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bac8f8-fbbd-4cc2-a862-d8b4a0ad865c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:33.568087 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571191f-2291-48c6-837d-7f99e0311b2c None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:33.612030 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738be877-b7c3-4860-84d5-85d0916e8f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:33.612696 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738be877-b7c3-4860-84d5-85d0916e8f90 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:33.613026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738be877-b7c3-4860-84d5-85d0916e8f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:33.613309 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738be877-b7c3-4860-84d5-85d0916e8f90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:33.633737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738be877-b7c3-4860-84d5-85d0916e8f90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:33.645300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-738be877-b7c3-4860-84d5-85d0916e8f90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:46:33.645300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:33.645300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:33.645300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:33.645300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:33.645300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:33.645300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:33.645300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:33.645300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:33.645300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:33.645300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:33.645300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:33.645300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:33.645300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:33.645300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:33.645300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:33.646457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:33.646457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:33.646457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:33.646457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:33.646457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:33.646457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:33.646457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:33.646457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:33.646457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:33.646457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:33.646457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:33.646457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:33.646457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:33.646457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:33.646457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:33.646457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:33.647463 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:33.647463 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:33.647463 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:33.647463 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:33.647463 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:33.647463 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:33.647463 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:33.647463 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:33.647463 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:33.647463 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:33.647463 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:46:33.647463 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:33.648120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 225/438] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:46:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:33.655848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738be877-b7c3-4860-84d5-85d0916e8f90 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:33.662387 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:33.680055 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bac8f8-fbbd-4cc2-a862-d8b4a0ad865c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:33.719522 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb6116-8246-4717-8795-8b9f807834b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:33.720755 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb6116-8246-4717-8795-8b9f807834b5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:33.721186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb6116-8246-4717-8795-8b9f807834b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:33.721601 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb6116-8246-4717-8795-8b9f807834b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:33.797928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738be877-b7c3-4860-84d5-85d0916e8f90 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:33.806982 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:33.807257 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb6116-8246-4717-8795-8b9f807834b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:33.808787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb6116-8246-4717-8795-8b9f807834b5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:33.838080 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a525678-15c6-4bcd-80b8-cc96f9b9af18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:33.838813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a525678-15c6-4bcd-80b8-cc96f9b9af18 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:33.839105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a525678-15c6-4bcd-80b8-cc96f9b9af18 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:33.839536 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a525678-15c6-4bcd-80b8-cc96f9b9af18 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:33.840324 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a525678-15c6-4bcd-80b8-cc96f9b9af18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:33.840662 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a525678-15c6-4bcd-80b8-cc96f9b9af18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:33.842275 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a525678-15c6-4bcd-80b8-cc96f9b9af18 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:33.857470 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 226/439] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:33.860231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38a09c-d580-41d2-ba7b-fbc331243ecd None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:33.860231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38a09c-d580-41d2-ba7b-fbc331243ecd None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:33.860402 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38a09c-d580-41d2-ba7b-fbc331243ecd None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:33.862664 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 227/440] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:33.863486 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38a09c-d580-41d2-ba7b-fbc331243ecd None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:33.914924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:33.925751 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 214/441] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:46:33.927016 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c87fe8-c067-42cd-a44d-c7b25a1ea38d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:33.927300 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c87fe8-c067-42cd-a44d-c7b25a1ea38d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:33.927579 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c87fe8-c067-42cd-a44d-c7b25a1ea38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:33.945151 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-95c87fe8-c067-42cd-a44d-c7b25a1ea38d None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:33.946410 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-95c87fe8-c067-42cd-a44d-c7b25a1ea38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:33.969808 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38a09c-d580-41d2-ba7b-fbc331243ecd None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:34.006547 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496b731-8fbc-4ce8-b85f-077544c9a2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:34.007299 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496b731-8fbc-4ce8-b85f-077544c9a2dd None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:34.007423 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496b731-8fbc-4ce8-b85f-077544c9a2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:34.007739 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496b731-8fbc-4ce8-b85f-077544c9a2dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:34.020115 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 215/442] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:34.021007 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c87fe8-c067-42cd-a44d-c7b25a1ea38d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:34.048711 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:34.084058 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496b731-8fbc-4ce8-b85f-077544c9a2dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:34.085517 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496b731-8fbc-4ce8-b85f-077544c9a2dd None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:34.129512 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c87fe8-c067-42cd-a44d-c7b25a1ea38d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:34.149489 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:34.164240 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9bdb2c-e66a-4360-9e18-b86c7584f7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:34.164885 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9bdb2c-e66a-4360-9e18-b86c7584f7ef None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:34.165215 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9bdb2c-e66a-4360-9e18-b86c7584f7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:34.165435 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9bdb2c-e66a-4360-9e18-b86c7584f7ef None admin] PATH_INFO: `/v3/projects/fce7a22bc8ff438d90035074a2c73590` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:34.166582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9bdb2c-e66a-4360-9e18-b86c7584f7ef None admin] RBAC: Authorizing `identity:get_project(project_id=fce7a22bc8ff438d90035074a2c73590)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:34.168076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9bdb2c-e66a-4360-9e18-b86c7584f7ef None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:34.169805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 216/443] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:34 2022] GET /identity/v3/projects/fce7a22bc8ff438d90035074a2c73590 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:34.170556 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9bdb2c-e66a-4360-9e18-b86c7584f7ef None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:34.190890 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 228/444] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:46:34.191761 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496b731-8fbc-4ce8-b85f-077544c9a2dd None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:34.257660 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:34.269870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9bdb2c-e66a-4360-9e18-b86c7584f7ef None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:34.290410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496b731-8fbc-4ce8-b85f-077544c9a2dd None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:34.303019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ddb96b-b57f-4117-afbc-0bfff7465831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:34.303688 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ddb96b-b57f-4117-afbc-0bfff7465831 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:34.303980 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ddb96b-b57f-4117-afbc-0bfff7465831 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:34.304252 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ddb96b-b57f-4117-afbc-0bfff7465831 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:34.314450 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ddb96b-b57f-4117-afbc-0bfff7465831 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:34.319438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e9ddb96b-b57f-4117-afbc-0bfff7465831 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 04:46:34.319438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:34.319438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:34.319438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:34.319438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:34.319438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:34.319438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:34.319438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:34.319438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:34.319438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:34.319438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:34.319438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:34.319438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:34.319438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:34.319438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:34.319438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:34.320434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:34.320434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:34.320434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:34.320434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:34.320434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:34.320434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:34.320434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:34.320434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:34.320434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:34.320434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:34.320434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:34.320434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:34.320434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:34.320434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:34.320434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:34.320434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:34.321435 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:34.321435 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:34.321435 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:34.321435 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:34.321435 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:34.321435 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:34.321435 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:34.321435 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:34.321435 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:34.321435 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:34.321435 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 04:46:34.321435 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:34.321435 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 217/445] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jul 26 04:46:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:34.322350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ddb96b-b57f-4117-afbc-0bfff7465831 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:34.328578 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93618d64-052d-4ddb-8e34-66bbb4ea7a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:34.329245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93618d64-052d-4ddb-8e34-66bbb4ea7a0d None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:34.329525 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93618d64-052d-4ddb-8e34-66bbb4ea7a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:34.329799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93618d64-052d-4ddb-8e34-66bbb4ea7a0d None admin] PATH_INFO: `/v3/groups/a5b3ed6cc57044e19dfae5d635209526` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:34.330982 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93618d64-052d-4ddb-8e34-66bbb4ea7a0d None admin] RBAC: Authorizing `identity:get_group(group_id=a5b3ed6cc57044e19dfae5d635209526)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:34.332496 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93618d64-052d-4ddb-8e34-66bbb4ea7a0d None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:34.334118 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 229/446] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jul 26 04:46:34 2022] GET /identity/v3/groups/a5b3ed6cc57044e19dfae5d635209526 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:34.334886 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93618d64-052d-4ddb-8e34-66bbb4ea7a0d None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:34.380122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:34.433875 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ddb96b-b57f-4117-afbc-0bfff7465831 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:34.438529 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93618d64-052d-4ddb-8e34-66bbb4ea7a0d None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:34.477165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8c699-6039-4367-90a9-e2b4830e4fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:34.477574 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81f86c-1a90-4342-8314-78e671d401f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:34.477798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8c699-6039-4367-90a9-e2b4830e4fdf None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:34.478090 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8c699-6039-4367-90a9-e2b4830e4fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:34.478218 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81f86c-1a90-4342-8314-78e671d401f1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:34.478366 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8c699-6039-4367-90a9-e2b4830e4fdf None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:34.478508 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81f86c-1a90-4342-8314-78e671d401f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:34.478786 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81f86c-1a90-4342-8314-78e671d401f1 None admin] PATH_INFO: `/v3/projects/d12823499382429e9825c57ea3292c56` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:34.479104 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e8c699-6039-4367-90a9-e2b4830e4fdf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:34.479413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e8c699-6039-4367-90a9-e2b4830e4fdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:34.479995 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81f86c-1a90-4342-8314-78e671d401f1 None admin] RBAC: Authorizing `identity:get_project(project_id=d12823499382429e9825c57ea3292c56)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:34.480922 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e8c699-6039-4367-90a9-e2b4830e4fdf None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:34.481474 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81f86c-1a90-4342-8314-78e671d401f1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:34.483269 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 218/447] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:34 2022] GET /identity/v3/projects/d12823499382429e9825c57ea3292c56 => generated 344 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:34.483393 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:34.483947 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81f86c-1a90-4342-8314-78e671d401f1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:34.494123 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 230/448] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:46:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:34.494975 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8c699-6039-4367-90a9-e2b4830e4fdf None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:34.588860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81f86c-1a90-4342-8314-78e671d401f1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:34.595470 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8c699-6039-4367-90a9-e2b4830e4fdf None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:34.607531 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:34.631916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d9731-a96c-4b71-8112-bb003687e4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:34.632850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d9731-a96c-4b71-8112-bb003687e4df None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:34.633133 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d9731-a96c-4b71-8112-bb003687e4df None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:34.633406 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d9731-a96c-4b71-8112-bb003687e4df None admin] PATH_INFO: `/v3/groups/d28be20a3a1441e59d04b97496814a5a` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:34.633531 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e63b5-1711-429c-a3cd-13ff78dc0971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:34.634481 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e63b5-1711-429c-a3cd-13ff78dc0971 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:34.634692 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d9731-a96c-4b71-8112-bb003687e4df None admin] RBAC: Authorizing `identity:get_group(group_id=d28be20a3a1441e59d04b97496814a5a)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:34.634909 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e63b5-1711-429c-a3cd-13ff78dc0971 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:34.635283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e63b5-1711-429c-a3cd-13ff78dc0971 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:34.636216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d9731-a96c-4b71-8112-bb003687e4df None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:34.636587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e63b5-1711-429c-a3cd-13ff78dc0971 None admin] RBAC: Adding query filter params (group.id=a5b3ed6cc57044e19dfae5d635209526, role.id=7519356859fb40508c25bec8161207fd, scope.project.id=fce7a22bc8ff438d90035074a2c73590) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:34.637068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e63b5-1711-429c-a3cd-13ff78dc0971 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:34.637955 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 231/449] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jul 26 04:46:34 2022] GET /identity/v3/groups/d28be20a3a1441e59d04b97496814a5a => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:34.638733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d9731-a96c-4b71-8112-bb003687e4df None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:34.639602 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e63b5-1711-429c-a3cd-13ff78dc0971 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:34.655630 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 219/450] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Jul 26 04:46:34 2022] GET /identity/v3/role_assignments?group.id=a5b3ed6cc57044e19dfae5d635209526&scope.project.id=fce7a22bc8ff438d90035074a2c73590&role.id=7519356859fb40508c25bec8161207fd => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:34.655630 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e63b5-1711-429c-a3cd-13ff78dc0971 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:34.729102 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:34.772583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d9731-a96c-4b71-8112-bb003687e4df None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:34.775851 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e63b5-1711-429c-a3cd-13ff78dc0971 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:34.812013 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c1144-423c-41e5-b1f1-afe5197954a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:34.812652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c1144-423c-41e5-b1f1-afe5197954a1 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:34.812938 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c1144-423c-41e5-b1f1-afe5197954a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:34.813212 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c1144-423c-41e5-b1f1-afe5197954a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:34.816071 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c5884-b1f6-41b3-ace8-6f1d612f4617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:34.816700 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c5884-b1f6-41b3-ace8-6f1d612f4617 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:34.817003 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c5884-b1f6-41b3-ace8-6f1d612f4617 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:34.817259 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c5884-b1f6-41b3-ace8-6f1d612f4617 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:34.818056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c5884-b1f6-41b3-ace8-6f1d612f4617 None admin] RBAC: Adding query filter params (group.id=d28be20a3a1441e59d04b97496814a5a, role.id=72bf8bd23d2c4b76ad32b7b35d6db412, scope.project.id=d12823499382429e9825c57ea3292c56) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:34.818359 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c5884-b1f6-41b3-ace8-6f1d612f4617 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:34.819850 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c5884-b1f6-41b3-ace8-6f1d612f4617 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:34.826422 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 220/451] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Jul 26 04:46:34 2022] GET /identity/v3/role_assignments?group.id=d28be20a3a1441e59d04b97496814a5a&scope.project.id=d12823499382429e9825c57ea3292c56&role.id=72bf8bd23d2c4b76ad32b7b35d6db412 => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:34.902011 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c1144-423c-41e5-b1f1-afe5197954a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:34.904502 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c1144-423c-41e5-b1f1-afe5197954a1 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:34.982636 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:35.042140 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 232/452] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:46:35.048299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c5884-b1f6-41b3-ace8-6f1d612f4617 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:35.102042 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:35.160888 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c5884-b1f6-41b3-ace8-6f1d612f4617 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:35.197995 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f04a1-2000-470a-bd29-7aeba7b64abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:35.198680 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f04a1-2000-470a-bd29-7aeba7b64abd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:35.199037 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f04a1-2000-470a-bd29-7aeba7b64abd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:35.199302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f04a1-2000-470a-bd29-7aeba7b64abd None admin] PATH_INFO: `/v3/projects/fce7a22bc8ff438d90035074a2c73590` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:35.200562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f04a1-2000-470a-bd29-7aeba7b64abd None admin] RBAC: Authorizing `identity:get_project(project_id=fce7a22bc8ff438d90035074a2c73590)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:35.202021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f04a1-2000-470a-bd29-7aeba7b64abd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:35.203844 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 221/453] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:46:35 2022] GET /identity/v3/projects/fce7a22bc8ff438d90035074a2c73590 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:35.209964 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c1144-423c-41e5-b1f1-afe5197954a1 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:35.312910 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c1144-423c-41e5-b1f1-afe5197954a1 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:35.350528 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961e1a5-d756-4460-a872-8c27986ae593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:35.351179 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961e1a5-d756-4460-a872-8c27986ae593 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:35.351459 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961e1a5-d756-4460-a872-8c27986ae593 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:35.351764 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961e1a5-d756-4460-a872-8c27986ae593 None admin] PATH_INFO: `/v3/groups/a5b3ed6cc57044e19dfae5d635209526` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:35.353052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961e1a5-d756-4460-a872-8c27986ae593 None admin] RBAC: Authorizing `identity:get_group(group_id=a5b3ed6cc57044e19dfae5d635209526)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:35.354524 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961e1a5-d756-4460-a872-8c27986ae593 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:35.356173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 233/454] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jul 26 04:46:35 2022] GET /identity/v3/groups/a5b3ed6cc57044e19dfae5d635209526 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:35.362482 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f04a1-2000-470a-bd29-7aeba7b64abd None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:35.471299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f04a1-2000-470a-bd29-7aeba7b64abd None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:35.484053 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:35.505900 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf03f9b-069d-4014-af56-e77d3f05f3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:35.506526 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf03f9b-069d-4014-af56-e77d3f05f3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:35.506807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf03f9b-069d-4014-af56-e77d3f05f3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:35.507075 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf03f9b-069d-4014-af56-e77d3f05f3a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:35.507898 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf03f9b-069d-4014-af56-e77d3f05f3a2 None admin] RBAC: Adding query filter params (group.id=a5b3ed6cc57044e19dfae5d635209526, role.id=9223a9bb40484faca3a7bcfeb41a2215, scope.project.id=fce7a22bc8ff438d90035074a2c73590) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:35.508204 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf03f9b-069d-4014-af56-e77d3f05f3a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:35.509646 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf03f9b-069d-4014-af56-e77d3f05f3a2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:35.515898 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 222/455] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Jul 26 04:46:35 2022] GET /identity/v3/role_assignments?group.id=a5b3ed6cc57044e19dfae5d635209526&scope.project.id=fce7a22bc8ff438d90035074a2c73590&role.id=9223a9bb40484faca3a7bcfeb41a2215 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:38.124069 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01286caa-c9f0-46a0-b862-066efb060261 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:38.124627 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01286caa-c9f0-46a0-b862-066efb060261 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:38.124627 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01286caa-c9f0-46a0-b862-066efb060261 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:38.126197 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 234/456] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:38.131691 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7b685-92eb-476d-8ad4-5b7bdeec3995 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:38.132116 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7b685-92eb-476d-8ad4-5b7bdeec3995 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:38.132217 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7b685-92eb-476d-8ad4-5b7bdeec3995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:38.156019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7fa7b685-92eb-476d-8ad4-5b7bdeec3995 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:38.157574 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7fa7b685-92eb-476d-8ad4-5b7bdeec3995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:38.258165 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 223/457] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:38.302324 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01286caa-c9f0-46a0-b862-066efb060261 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:38.366498 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:38.461245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01286caa-c9f0-46a0-b862-066efb060261 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:38.510250 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a752ba-4965-4227-a252-cc21b1d9b2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:38.511171 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a752ba-4965-4227-a252-cc21b1d9b2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:38.511623 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a752ba-4965-4227-a252-cc21b1d9b2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:38.512065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a752ba-4965-4227-a252-cc21b1d9b2f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:38.527023 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a752ba-4965-4227-a252-cc21b1d9b2f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:38.528806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a752ba-4965-4227-a252-cc21b1d9b2f2 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:38.551189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-04a752ba-4965-4227-a252-cc21b1d9b2f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:38.551189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:38.551189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:38.551189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:38.551189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:38.551189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:38.551189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:38.551189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:38.551189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:38.551189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:38.551189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:38.551189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:38.551189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:38.551189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:38.551189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:38.552345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:38.552345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:38.552345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:38.552345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:38.552345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:38.552345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:38.552345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:38.552345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:38.552345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:38.552345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:38.552345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:38.552345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:38.552345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:38.552345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:38.552345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:38.552345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:38.552345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:38.553417 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:38.553417 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:38.553417 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:38.553417 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:38.553699 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28dd37-dece-4a83-bec4-3adcc62e3979 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:38.553699 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28dd37-dece-4a83-bec4-3adcc62e3979 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:38.553699 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28dd37-dece-4a83-bec4-3adcc62e3979 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:38.554135 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 224/458] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:38.555644 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 235/459] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:38.563609 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36665ad-7dd4-4317-98a8-b3b5fb032901 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:38.563800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c90f5-d4d8-4fa9-a30a-1c1498496b3f None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:38.563890 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36665ad-7dd4-4317-98a8-b3b5fb032901 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:38.563972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c90f5-d4d8-4fa9-a30a-1c1498496b3f None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:38.564071 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36665ad-7dd4-4317-98a8-b3b5fb032901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:38.564152 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c90f5-d4d8-4fa9-a30a-1c1498496b3f None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:38.564152 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 236/460] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:38.565295 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c90f5-d4d8-4fa9-a30a-1c1498496b3f None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:38.584899 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f36665ad-7dd4-4317-98a8-b3b5fb032901 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:38.586424 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f36665ad-7dd4-4317-98a8-b3b5fb032901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:38.658637 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 225/461] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:38.660352 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2601ba-65b4-412d-aba6-b76508fb9e8f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:38.660818 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2601ba-65b4-412d-aba6-b76508fb9e8f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:38.661270 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2601ba-65b4-412d-aba6-b76508fb9e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:38.671414 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:38.685196 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5e2601ba-65b4-412d-aba6-b76508fb9e8f None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:38.687752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5e2601ba-65b4-412d-aba6-b76508fb9e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:38.704599 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c90f5-d4d8-4fa9-a30a-1c1498496b3f None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:38.729324 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:38.748986 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f9642b-8e66-4ebc-95bc-84577c43b698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:38.750268 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9642b-8e66-4ebc-95bc-84577c43b698 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:38.751021 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9642b-8e66-4ebc-95bc-84577c43b698 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:38.751448 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9642b-8e66-4ebc-95bc-84577c43b698 None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:38.752663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f9642b-8e66-4ebc-95bc-84577c43b698 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:38.753116 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f9642b-8e66-4ebc-95bc-84577c43b698 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:38.755364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f9642b-8e66-4ebc-95bc-84577c43b698 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:38.771374 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 237/462] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:38.773515 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f9642b-8e66-4ebc-95bc-84577c43b698 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:38.780281 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 226/463] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:38.781416 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2601ba-65b4-412d-aba6-b76508fb9e8f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:38.882325 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2601ba-65b4-412d-aba6-b76508fb9e8f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:38.913942 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f9642b-8e66-4ebc-95bc-84577c43b698 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:38.917542 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9aa90a-a286-4b76-92a7-834ce7c25312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:38.918178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9aa90a-a286-4b76-92a7-834ce7c25312 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:38.918546 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9aa90a-a286-4b76-92a7-834ce7c25312 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:38.918914 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9aa90a-a286-4b76-92a7-834ce7c25312 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:38.919631 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9aa90a-a286-4b76-92a7-834ce7c25312 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:38.921138 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9aa90a-a286-4b76-92a7-834ce7c25312 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:38.944364 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 227/464] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jul 26 04:46:38 2022] POST /identity/v3/users => generated 246 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:46:38.945209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9aa90a-a286-4b76-92a7-834ce7c25312 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:38.960352 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b31f8-2093-4dce-ab17-62905bb292a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:38.961456 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b31f8-2093-4dce-ab17-62905bb292a9 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:38.962082 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b31f8-2093-4dce-ab17-62905bb292a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:38.962430 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b31f8-2093-4dce-ab17-62905bb292a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:38.976438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b31f8-2093-4dce-ab17-62905bb292a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:38.978122 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b31f8-2093-4dce-ab17-62905bb292a9 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:38.979524 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:38.989100 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:38.999813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-248b31f8-2093-4dce-ab17-62905bb292a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:38.999813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:38.999813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:38.999813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:38.999813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:38.999813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:38.999813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:38.999813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:38.999813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:38.999813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:38.999813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:38.999813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:38.999813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:38.999813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:38.999813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:39.000765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:39.000765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:39.000765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:39.000765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:39.000765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:39.000765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:39.000765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:39.000765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:39.000765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:39.000765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:39.000765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:39.000765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:39.000765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:39.000765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:39.000765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:39.000765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:39.000765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:39.001688 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:39.001688 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:39.001688 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:39.001688 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:39.008451 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 238/465] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:39.008451 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03415bb1-a5c9-458e-b9d3-00561d5ba0d6 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:39.008451 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03415bb1-a5c9-458e-b9d3-00561d5ba0d6 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:39.008451 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03415bb1-a5c9-458e-b9d3-00561d5ba0d6 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:39.008451 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 239/466] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:39.013455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03415bb1-a5c9-458e-b9d3-00561d5ba0d6 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:39.102361 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9aa90a-a286-4b76-92a7-834ce7c25312 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:39.146568 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a0e94-198e-4405-808c-17dbf2f63133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:39.147241 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a0e94-198e-4405-808c-17dbf2f63133 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:39.147509 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a0e94-198e-4405-808c-17dbf2f63133 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:39.147815 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a0e94-198e-4405-808c-17dbf2f63133 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:39.148649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03415bb1-a5c9-458e-b9d3-00561d5ba0d6 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:39.160365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a0e94-198e-4405-808c-17dbf2f63133 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:39.161971 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a0e94-198e-4405-808c-17dbf2f63133 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:39.181376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-1c0a0e94-198e-4405-808c-17dbf2f63133 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:39.181376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:39.181376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:39.181376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:39.181376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:39.181376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:39.181376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:39.181376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:39.181376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:39.181376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:39.181376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:39.181376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:39.181376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:39.181376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:39.181376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:39.182305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:39.182305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:39.182305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:39.182305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:39.182305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:39.182305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:39.182305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:39.182305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:39.182305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:39.182305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:39.182305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:39.182305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:39.182305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:39.182305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:39.182305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:39.182305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:39.182305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:39.183662 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:39.183662 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:39.183662 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:39.183662 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:39.183662 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 228/467] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:39.184165 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596bfa4-1692-40d7-8d03-5757551a669d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:39.184165 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596bfa4-1692-40d7-8d03-5757551a669d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:39.184325 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596bfa4-1692-40d7-8d03-5757551a669d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:39.195511 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99891f3c-daf2-4bd3-b4ad-cf4eea37edc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:39.196184 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99891f3c-daf2-4bd3-b4ad-cf4eea37edc7 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:39.196476 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99891f3c-daf2-4bd3-b4ad-cf4eea37edc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:39.196746 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99891f3c-daf2-4bd3-b4ad-cf4eea37edc7 None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:39.197459 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99891f3c-daf2-4bd3-b4ad-cf4eea37edc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:39.197772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99891f3c-daf2-4bd3-b4ad-cf4eea37edc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:39.199162 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99891f3c-daf2-4bd3-b4ad-cf4eea37edc7 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:39.204810 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c596bfa4-1692-40d7-8d03-5757551a669d None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:39.206594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c596bfa4-1692-40d7-8d03-5757551a669d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:39.214112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 240/468] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:39.214803 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99891f3c-daf2-4bd3-b4ad-cf4eea37edc7 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:39.290761 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:39.302495 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 229/469] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:39.303347 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596bfa4-1692-40d7-8d03-5757551a669d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:39.325202 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:39.359244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99891f3c-daf2-4bd3-b4ad-cf4eea37edc7 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:39.405847 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3f660-0af4-4e26-8bf6-628ac7f137c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:39.406768 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3f660-0af4-4e26-8bf6-628ac7f137c6 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:39.407196 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3f660-0af4-4e26-8bf6-628ac7f137c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:39.407648 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3f660-0af4-4e26-8bf6-628ac7f137c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:39.408505 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c3f660-0af4-4e26-8bf6-628ac7f137c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:39.408956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c3f660-0af4-4e26-8bf6-628ac7f137c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:39.410516 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c3f660-0af4-4e26-8bf6-628ac7f137c6 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:39.424251 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 241/470] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:39.425495 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3f660-0af4-4e26-8bf6-628ac7f137c6 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:39.462131 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596bfa4-1692-40d7-8d03-5757551a669d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:39.509773 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e99978-c577-44d8-bd1d-73013b973e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:39.510742 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e99978-c577-44d8-bd1d-73013b973e8d None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:39.511185 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e99978-c577-44d8-bd1d-73013b973e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:39.511471 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e99978-c577-44d8-bd1d-73013b973e8d None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:39.512206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e99978-c577-44d8-bd1d-73013b973e8d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:39.513746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e99978-c577-44d8-bd1d-73013b973e8d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:39.535164 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3f660-0af4-4e26-8bf6-628ac7f137c6 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:39.537594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 230/471] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jul 26 04:46:39 2022] POST /identity/v3/users => generated 248 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:46:39.538457 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e99978-c577-44d8-bd1d-73013b973e8d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:39.570553 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:39.592156 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d33f5-579d-472e-a02c-08437d379513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:39.593712 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d33f5-579d-472e-a02c-08437d379513 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:39.594367 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d33f5-579d-472e-a02c-08437d379513 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:39.595030 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d33f5-579d-472e-a02c-08437d379513 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:39.613313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d33f5-579d-472e-a02c-08437d379513 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:39.615162 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d33f5-579d-472e-a02c-08437d379513 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:39.624926 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:39.637054 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-696d33f5-579d-472e-a02c-08437d379513 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:39.637054 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:39.637054 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:39.637054 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:39.637054 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:39.637054 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:39.637054 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:39.637054 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:39.637054 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:39.637054 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:39.637054 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:39.637054 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:39.637054 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:39.637054 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:39.637054 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:39.638038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:39.638038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:39.638038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:39.638038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:39.638038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:39.638038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:39.638038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:39.638038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:39.638038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:39.638038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:39.638038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:39.638038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:39.638038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:39.638038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:39.638038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:39.638038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:39.638038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:39.638937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:39.638937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:39.638937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:39.638937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:39.641195 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 242/472] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:39.642770 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f1206-e95a-4f12-be33-8a592cf24057 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:39.643215 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f1206-e95a-4f12-be33-8a592cf24057 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:39.643642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f1206-e95a-4f12-be33-8a592cf24057 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:39.649482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 243/473] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:39.650861 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f1206-e95a-4f12-be33-8a592cf24057 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:39.693303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e99978-c577-44d8-bd1d-73013b973e8d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:39.751524 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b6904-e6c5-410a-a8f1-0167f1cd6cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:39.752756 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b6904-e6c5-410a-a8f1-0167f1cd6cf5 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:39.753191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b6904-e6c5-410a-a8f1-0167f1cd6cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:39.753469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b6904-e6c5-410a-a8f1-0167f1cd6cf5 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:39.754167 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b6904-e6c5-410a-a8f1-0167f1cd6cf5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:39.755701 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b6904-e6c5-410a-a8f1-0167f1cd6cf5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:39.782715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 231/474] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jul 26 04:46:39 2022] POST /identity/v3/users => generated 248 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:46:39.784057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1dc62-9830-4fa2-9eff-ce6310a78ca6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:39.784165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f1206-e95a-4f12-be33-8a592cf24057 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:39.784332 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1dc62-9830-4fa2-9eff-ce6310a78ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:39.784606 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1dc62-9830-4fa2-9eff-ce6310a78ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:39.805104 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-38a1dc62-9830-4fa2-9eff-ce6310a78ca6 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:39.806468 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-38a1dc62-9830-4fa2-9eff-ce6310a78ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:39.828754 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a0603-c257-4d0b-bd39-767c41d18061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:39.829445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a0603-c257-4d0b-bd39-767c41d18061 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:39.829777 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a0603-c257-4d0b-bd39-767c41d18061 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:39.830099 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a0603-c257-4d0b-bd39-767c41d18061 None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:39.830976 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a0603-c257-4d0b-bd39-767c41d18061 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:39.831287 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a0603-c257-4d0b-bd39-767c41d18061 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:39.832841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a0603-c257-4d0b-bd39-767c41d18061 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:39.848202 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 244/475] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:39.854203 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a0603-c257-4d0b-bd39-767c41d18061 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:39.885526 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 232/476] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:39.902017 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:39.921017 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1dc62-9830-4fa2-9eff-ce6310a78ca6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:39.978054 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:40.006518 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a0603-c257-4d0b-bd39-767c41d18061 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:40.044401 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1dc62-9830-4fa2-9eff-ce6310a78ca6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:40.056524 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a32013-e94f-4c17-a7b2-bd0a720e3ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:40.057929 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a32013-e94f-4c17-a7b2-bd0a720e3ef1 None admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:40.058464 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a32013-e94f-4c17-a7b2-bd0a720e3ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:40.058913 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a32013-e94f-4c17-a7b2-bd0a720e3ef1 None admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:40.060006 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a32013-e94f-4c17-a7b2-bd0a720e3ef1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:40.062321 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a32013-e94f-4c17-a7b2-bd0a720e3ef1 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:40.091308 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c97220-5d5d-4571-a918-e92b3e5eb9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:40.092156 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c97220-5d5d-4571-a918-e92b3e5eb9d6 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:40.092541 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c97220-5d5d-4571-a918-e92b3e5eb9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:40.093340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c97220-5d5d-4571-a918-e92b3e5eb9d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:40.094256 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c97220-5d5d-4571-a918-e92b3e5eb9d6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:40.096075 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c97220-5d5d-4571-a918-e92b3e5eb9d6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:40.097400 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 245/477] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jul 26 04:46:39 2022] POST /identity/v3/users => generated 249 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 04:46:40.110980 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.sql.core [None req-f4c97220-5d5d-4571-a918-e92b3e5eb9d6 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 26 04:46:40.110980 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 26 04:46:40.110980 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [parameters: {'id': 'ab73c9c9b2664b7bb2681b89917afdb9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 26 04:46:40.110980 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80206) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 26 04:46:40.128182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-f4c97220-5d5d-4571-a918-e92b3e5eb9d6 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 26 04:46:40.128182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:40.128182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 04:46:40.128182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 04:46:40.128182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 04:46:40.128182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 04:46:40.128182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 04:46:40.128182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 04:46:40.128182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 04:46:40.128182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application conn.query(q) Jul 26 04:46:40.128182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 04:46:40.128182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 04:46:40.128182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 04:46:40.128182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result.read() Jul 26 04:46:40.128182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 04:46:40.129126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 04:46:40.129126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 04:46:40.129126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 04:46:40.129126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 04:46:40.129126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 04:46:40.129126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 04:46:40.129126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 04:46:40.129126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 26 04:46:40.129126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:40.129126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 26 04:46:40.129126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:40.129126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:40.129126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 26 04:46:40.129126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 26 04:46:40.129126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 26 04:46:40.129126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return ref.to_dict() Jul 26 04:46:40.129126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 04:46:40.129126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application next(self.gen) Jul 26 04:46:40.130129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 26 04:46:40.130129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application yield resource Jul 26 04:46:40.130129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 04:46:40.130129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application next(self.gen) Jul 26 04:46:40.130129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 26 04:46:40.130129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.session.rollback() Jul 26 04:46:40.130129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 26 04:46:40.130129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.force_reraise() Jul 26 04:46:40.130129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 26 04:46:40.130129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise self.value Jul 26 04:46:40.130129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 26 04:46:40.130129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 26 04:46:40.130129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 26 04:46:40.130129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application session.commit() Jul 26 04:46:40.130129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 26 04:46:40.130129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 26 04:46:40.130129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 26 04:46:40.130943 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._prepare_impl() Jul 26 04:46:40.130943 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 26 04:46:40.130943 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.session.flush() Jul 26 04:46:40.130943 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 26 04:46:40.130943 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._flush(objects) Jul 26 04:46:40.130943 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 26 04:46:40.130943 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 26 04:46:40.130943 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 26 04:46:40.130943 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application compat.raise_( Jul 26 04:46:40.130943 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 04:46:40.130943 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception Jul 26 04:46:40.130943 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 26 04:46:40.130943 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application flush_context.execute() Jul 26 04:46:40.130943 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 26 04:46:40.130943 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rec.execute(self) Jul 26 04:46:40.130943 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 26 04:46:40.130943 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 26 04:46:40.131897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 26 04:46:40.131897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 26 04:46:40.131897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 26 04:46:40.131897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 26 04:46:40.131897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 26 04:46:40.131897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 26 04:46:40.131897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 26 04:46:40.131897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 26 04:46:40.131897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 26 04:46:40.131897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 26 04:46:40.131897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 26 04:46:40.131897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 26 04:46:40.131897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 26 04:46:40.131897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 26 04:46:40.131897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 04:46:40.131897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception Jul 26 04:46:40.132708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 04:46:40.132708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 04:46:40.132708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 04:46:40.132708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 04:46:40.132708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 04:46:40.132708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 04:46:40.132708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 04:46:40.132708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application conn.query(q) Jul 26 04:46:40.132708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 04:46:40.132708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 04:46:40.132708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 04:46:40.132708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result.read() Jul 26 04:46:40.132708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 04:46:40.132708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 04:46:40.132708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 04:46:40.132708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 04:46:40.133493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 04:46:40.133493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 04:46:40.133493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 04:46:40.133493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 04:46:40.133493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 26 04:46:40.133493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 26 04:46:40.133493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [parameters: {'id': 'ab73c9c9b2664b7bb2681b89917afdb9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 26 04:46:40.133493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 26 04:46:40.133493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:40.133493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:46:40.133493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:40.133493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:40.133493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:40.133493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:40.133493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:40.133493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:40.134296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:40.134296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:40.134296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:40.134296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:40.134296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:40.134296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:40.134296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 26 04:46:40.134296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 26 04:46:40.134296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:40.134296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:40.134296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 26 04:46:40.134296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 26 04:46:40.134296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 26 04:46:40.134296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 26 04:46:40.134296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 26 04:46:40.134296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:40.135131 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 233/478] 10.222.0.17 () {60 vars in 1131 bytes} [Tue Jul 26 04:46:39 2022] POST /identity/v3/roles => generated 144 bytes in 209 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:40.143050 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a32013-e94f-4c17-a7b2-bd0a720e3ef1 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:40.249972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:40.276476 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a32013-e94f-4c17-a7b2-bd0a720e3ef1 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:40.321110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6044d-a31a-415e-b38b-aadd1599fe17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:40.322180 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6044d-a31a-415e-b38b-aadd1599fe17 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:40.322661 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6044d-a31a-415e-b38b-aadd1599fe17 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:40.323132 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6044d-a31a-415e-b38b-aadd1599fe17 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:40.335816 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6044d-a31a-415e-b38b-aadd1599fe17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:40.343441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-aea6044d-a31a-415e-b38b-aadd1599fe17 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 04:46:40.343441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:40.343441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:40.343441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:40.343441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:40.343441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:40.343441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:40.343441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:40.343441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:40.343441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:40.343441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:40.343441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:40.343441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:40.343441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:40.343441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:40.343441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:40.344653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:40.344653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:40.344653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:40.344653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:40.344653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:40.344653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:40.344653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:40.344653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:40.344653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:40.344653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:40.344653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:40.344653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:40.344653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:40.344653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:40.344653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:40.344653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:40.345674 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:40.345674 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:40.345674 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:40.345674 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:40.345674 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:40.345674 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:40.345674 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:40.345674 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:40.345674 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:40.345674 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:40.345674 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 04:46:40.345674 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:40.349003 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 246/479] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jul 26 04:46:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:40.355361 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c97220-5d5d-4571-a918-e92b3e5eb9d6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:40.467090 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c97220-5d5d-4571-a918-e92b3e5eb9d6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:40.479523 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:40.505205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276e62b-287e-42ba-9bc4-d3c25f94a283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:40.506045 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276e62b-287e-42ba-9bc4-d3c25f94a283 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:40.506603 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276e62b-287e-42ba-9bc4-d3c25f94a283 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:40.507049 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276e62b-287e-42ba-9bc4-d3c25f94a283 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:40.507953 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276e62b-287e-42ba-9bc4-d3c25f94a283 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:40.508261 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276e62b-287e-42ba-9bc4-d3c25f94a283 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:40.509805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276e62b-287e-42ba-9bc4-d3c25f94a283 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:40.519258 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 234/480] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:46:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:41.251319 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465518d-2fcc-464a-851b-96be2d10d965 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:41.252966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465518d-2fcc-464a-851b-96be2d10d965 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:41.253778 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465518d-2fcc-464a-851b-96be2d10d965 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:41.256725 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 247/481] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:41.263307 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82946c-0dce-4076-a219-509288e55e2e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:41.264246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82946c-0dce-4076-a219-509288e55e2e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:41.265010 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82946c-0dce-4076-a219-509288e55e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:41.309035 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3c82946c-0dce-4076-a219-509288e55e2e None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:41.311903 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3c82946c-0dce-4076-a219-509288e55e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:41.466011 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 235/482] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:41.498194 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465518d-2fcc-464a-851b-96be2d10d965 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:41.686385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465518d-2fcc-464a-851b-96be2d10d965 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:41.740292 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9c1b1-2e80-40fb-b389-fbccd4a0e692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:41.740961 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9c1b1-2e80-40fb-b389-fbccd4a0e692 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:41.741255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9c1b1-2e80-40fb-b389-fbccd4a0e692 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:41.741534 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9c1b1-2e80-40fb-b389-fbccd4a0e692 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:41.756466 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb9c1b1-2e80-40fb-b389-fbccd4a0e692 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:41.761409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-fbb9c1b1-2e80-40fb-b389-fbccd4a0e692 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:41.761409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:41.761409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:41.761409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:41.761409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:41.761409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:41.761409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:41.761409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:41.761409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:41.761409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:41.761409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:41.761409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:41.761409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:41.761409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:41.761409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:41.761409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:41.762329 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:41.762329 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:41.762329 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:41.762329 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:41.762329 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:41.762329 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:41.762329 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:41.762329 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:41.762329 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:41.762329 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:41.762329 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:41.762329 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:41.762329 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:41.762329 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:41.762329 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:41.762329 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:41.763583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:41.763583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:41.763583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:41.763583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:41.763583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:41.763583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:41.763583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:41.763583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:41.763583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:41.763583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:41.763583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:41.763583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:41.763583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 248/483] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jul 26 04:46:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:41.772690 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82946c-0dce-4076-a219-509288e55e2e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:41.937574 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82946c-0dce-4076-a219-509288e55e2e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:41.939093 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:41.989115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6327c-d052-481c-8c43-3eddb953c6b1 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:41.989434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6327c-d052-481c-8c43-3eddb953c6b1 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:41.989680 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6327c-d052-481c-8c43-3eddb953c6b1 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:41.991288 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 249/484] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:46:41.994545 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59fa903-9567-4371-bfc4-7b7846eaa597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:41.995196 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59fa903-9567-4371-bfc4-7b7846eaa597 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:41.995522 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59fa903-9567-4371-bfc4-7b7846eaa597 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:41.995835 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59fa903-9567-4371-bfc4-7b7846eaa597 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:41.997573 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59fa903-9567-4371-bfc4-7b7846eaa597 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:41.997885 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59fa903-9567-4371-bfc4-7b7846eaa597 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:41.998075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0650ea-b126-4171-b59f-9e0ad4c1c122 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:41.998388 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0650ea-b126-4171-b59f-9e0ad4c1c122 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:41.998667 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0650ea-b126-4171-b59f-9e0ad4c1c122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:41.999302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59fa903-9567-4371-bfc4-7b7846eaa597 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:42.009063 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 236/485] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jul 26 04:46:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:42.014633 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59fa903-9567-4371-bfc4-7b7846eaa597 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:42.035880 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-8f0650ea-b126-4171-b59f-9e0ad4c1c122 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:42.036994 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:42.039777 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-8f0650ea-b126-4171-b59f-9e0ad4c1c122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:42.164415 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 250/486] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:42.181595 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59fa903-9567-4371-bfc4-7b7846eaa597 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:42.202370 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0650ea-b126-4171-b59f-9e0ad4c1c122 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:42.245298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb7d76-2217-48f5-8c17-be35c911c24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:42.246122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb7d76-2217-48f5-8c17-be35c911c24e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:42.246634 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb7d76-2217-48f5-8c17-be35c911c24e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:42.246920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb7d76-2217-48f5-8c17-be35c911c24e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:42.272778 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb7d76-2217-48f5-8c17-be35c911c24e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:42.274262 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb7d76-2217-48f5-8c17-be35c911c24e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:42.310858 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-87cb7d76-2217-48f5-8c17-be35c911c24e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:42.310858 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:42.310858 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:42.310858 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:42.310858 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:42.310858 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:42.310858 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:42.310858 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:42.310858 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:42.310858 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:42.310858 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:42.310858 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:42.310858 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:42.310858 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:42.310858 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:42.311978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:42.311978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:42.311978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:42.311978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:42.311978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:42.311978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:42.311978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:42.311978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:42.311978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:42.311978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:42.311978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:42.311978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:42.311978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:42.311978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:42.311978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:42.311978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:42.311978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:42.312989 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:42.312989 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:42.312989 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:42.312989 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:42.319329 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 237/487] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:46:42.320288 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb7d76-2217-48f5-8c17-be35c911c24e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:42.362518 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:42.379769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0650ea-b126-4171-b59f-9e0ad4c1c122 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:42.415586 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86851103-7211-4dcb-8538-60e430ed56c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:42.416254 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86851103-7211-4dcb-8538-60e430ed56c6 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:42.416537 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86851103-7211-4dcb-8538-60e430ed56c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:42.416814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86851103-7211-4dcb-8538-60e430ed56c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:42.420689 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:42.432364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86851103-7211-4dcb-8538-60e430ed56c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:42.439159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-86851103-7211-4dcb-8538-60e430ed56c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:42.439159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:42.439159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:42.439159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:42.439159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:42.439159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:42.439159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:42.439159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:42.439159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:42.439159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:42.439159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:42.439159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:42.439159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:42.439159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:42.439159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:42.439159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:42.440308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:42.440308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:42.440308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:42.440308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:42.440308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:42.440308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:42.440308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:42.440308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:42.440308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:42.440308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:42.440308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:42.440308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:42.440308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:42.440308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:42.440308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:42.440308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:42.441141 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:42.441141 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:42.441141 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:42.441141 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:42.441141 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:42.441141 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:42.441141 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:42.441141 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:42.441141 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:42.441141 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:42.441141 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:42.441141 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:42.441141 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 251/488] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jul 26 04:46:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:46:42.441994 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc2cf7-13c1-4862-9842-7eb7b80eabac None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:42.442302 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc2cf7-13c1-4862-9842-7eb7b80eabac None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:42.442586 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc2cf7-13c1-4862-9842-7eb7b80eabac None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:42.447960 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 252/489] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:42.447960 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc2cf7-13c1-4862-9842-7eb7b80eabac None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:42.476181 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb7d76-2217-48f5-8c17-be35c911c24e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:42.512295 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af96d71-216a-4c59-aad2-fdf266f2165a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:42.512955 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af96d71-216a-4c59-aad2-fdf266f2165a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:42.513233 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af96d71-216a-4c59-aad2-fdf266f2165a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:42.513517 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af96d71-216a-4c59-aad2-fdf266f2165a None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:42.514238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af96d71-216a-4c59-aad2-fdf266f2165a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:42.514548 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af96d71-216a-4c59-aad2-fdf266f2165a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:42.515991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af96d71-216a-4c59-aad2-fdf266f2165a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:42.539097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 238/490] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:42.539097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476dfc4-9dd0-4cdd-9c65-16fb65ac51b7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:42.539097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476dfc4-9dd0-4cdd-9c65-16fb65ac51b7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:42.539097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476dfc4-9dd0-4cdd-9c65-16fb65ac51b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:42.555117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d476dfc4-9dd0-4cdd-9c65-16fb65ac51b7 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:42.556364 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d476dfc4-9dd0-4cdd-9c65-16fb65ac51b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:42.572026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc2cf7-13c1-4862-9842-7eb7b80eabac None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:42.616727 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17887277-d3f9-4eb0-8e2b-9716e061a894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:42.631768 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17887277-d3f9-4eb0-8e2b-9716e061a894 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:42.631768 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17887277-d3f9-4eb0-8e2b-9716e061a894 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:42.631768 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17887277-d3f9-4eb0-8e2b-9716e061a894 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:42.631768 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17887277-d3f9-4eb0-8e2b-9716e061a894 None admin] RBAC: Adding query filter params (name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:42.631768 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17887277-d3f9-4eb0-8e2b-9716e061a894 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:42.631768 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17887277-d3f9-4eb0-8e2b-9716e061a894 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:42.634692 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 253/491] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jul 26 04:46:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:42.634692 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17887277-d3f9-4eb0-8e2b-9716e061a894 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:42.636968 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 239/492] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:42.639405 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476dfc4-9dd0-4cdd-9c65-16fb65ac51b7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:42.648803 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:42.699741 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:42.744992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17887277-d3f9-4eb0-8e2b-9716e061a894 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:42.752099 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476dfc4-9dd0-4cdd-9c65-16fb65ac51b7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:42.781759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22adbf2e-b57d-46fb-bcf0-1171e3396840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:42.782810 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22adbf2e-b57d-46fb-bcf0-1171e3396840 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:42.783297 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22adbf2e-b57d-46fb-bcf0-1171e3396840 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:42.783796 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22adbf2e-b57d-46fb-bcf0-1171e3396840 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:42.789880 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f6058-dda6-40fb-a1ab-4182f780c7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:42.790617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f6058-dda6-40fb-a1ab-4182f780c7f5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:42.790908 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f6058-dda6-40fb-a1ab-4182f780c7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:42.791189 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f6058-dda6-40fb-a1ab-4182f780c7f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:42.793570 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22adbf2e-b57d-46fb-bcf0-1171e3396840 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:42.795370 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22adbf2e-b57d-46fb-bcf0-1171e3396840 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:42.802681 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f6058-dda6-40fb-a1ab-4182f780c7f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:42.804254 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f6058-dda6-40fb-a1ab-4182f780c7f5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:42.809920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-22adbf2e-b57d-46fb-bcf0-1171e3396840 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 04:46:42.809920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:42.809920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:42.809920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:42.809920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:42.809920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:42.809920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:42.809920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:42.809920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:42.809920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:42.809920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:42.809920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:42.809920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:46:42.809920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:46:42.809920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:46:42.809920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:46:42.810879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:42.810879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:42.810879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:46:42.810879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:42.810879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:46:42.810879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:42.810879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:42.810879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:42.810879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:42.810879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:42.810879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:42.810879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:42.810879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:42.810879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:42.810879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:42.810879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:42.810879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:42.810879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:42.811856 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:42.811856 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:42.811856 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:46:42.811856 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:46:42.811856 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:46:42.811856 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:46:42.811856 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:46:42.811856 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:46:42.811856 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 04:46:42.811856 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:42.814643 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 254/493] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Jul 26 04:46:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:42.815811 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22adbf2e-b57d-46fb-bcf0-1171e3396840 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:42.820775 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-be3f6058-dda6-40fb-a1ab-4182f780c7f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:42.820775 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:42.820775 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:42.820775 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:42.820775 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:42.820775 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:42.820775 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:42.820775 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:42.820775 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:42.820775 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:42.820775 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:42.820775 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:42.820775 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:42.820775 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:42.820775 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:42.821799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:42.821799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:42.821799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:42.821799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:42.821799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:42.821799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:42.821799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:42.821799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:42.821799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:42.821799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:42.821799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:42.821799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:42.821799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:42.821799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:42.821799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:42.821799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:42.821799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:42.822635 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:42.822635 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:42.822635 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:42.822635 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:42.822635 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 240/494] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:42.823356 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f6058-dda6-40fb-a1ab-4182f780c7f5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:42.894844 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:42.927946 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22adbf2e-b57d-46fb-bcf0-1171e3396840 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:42.942079 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f6058-dda6-40fb-a1ab-4182f780c7f5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:42.947543 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:42.966338 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b601f-6518-4bcc-955c-20c6082044cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:42.967835 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b601f-6518-4bcc-955c-20c6082044cb None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:42.968331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b601f-6518-4bcc-955c-20c6082044cb None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:42.968807 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b601f-6518-4bcc-955c-20c6082044cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:42.987591 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b601f-6518-4bcc-955c-20c6082044cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:42.987778 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52567790-ae58-4c54-ad2a-acce6b89a36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:42.987778 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52567790-ae58-4c54-ad2a-acce6b89a36d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:42.987778 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52567790-ae58-4c54-ad2a-acce6b89a36d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:42.987778 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52567790-ae58-4c54-ad2a-acce6b89a36d None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:42.988083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52567790-ae58-4c54-ad2a-acce6b89a36d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:42.988083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52567790-ae58-4c54-ad2a-acce6b89a36d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:42.988083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52567790-ae58-4c54-ad2a-acce6b89a36d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:42.988266 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-d09b601f-6518-4bcc-955c-20c6082044cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:42.988266 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:42.988266 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:42.988266 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:42.988266 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:42.988266 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:42.988266 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:42.988266 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:42.988266 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:42.988266 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:42.988266 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:42.988266 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:42.988266 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:42.988266 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:42.988266 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:42.988266 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:42.989067 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:42.989067 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:42.989067 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:42.989067 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:42.989067 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:42.989067 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:42.989067 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:42.989067 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:42.989067 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:42.989067 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:42.989067 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:42.989067 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:42.989067 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:42.989067 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:42.989067 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:42.989067 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:42.989852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:42.989852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:42.989852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:42.989852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:42.989852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:42.989852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:42.989852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:42.989852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:42.989852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:42.989852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:42.989852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:42.989852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:42.989852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 255/495] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jul 26 04:46:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:42.990523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b601f-6518-4bcc-955c-20c6082044cb None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:43.031371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 241/496] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Jul 26 04:46:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:43.032360 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52567790-ae58-4c54-ad2a-acce6b89a36d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:43.108812 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b601f-6518-4bcc-955c-20c6082044cb None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:43.136128 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52567790-ae58-4c54-ad2a-acce6b89a36d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:43.144966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3c6fd-45fb-467b-816c-18b3e6cc9d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:43.146006 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3c6fd-45fb-467b-816c-18b3e6cc9d4e None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:43.146394 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3c6fd-45fb-467b-816c-18b3e6cc9d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:43.146741 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3c6fd-45fb-467b-816c-18b3e6cc9d4e None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:43.147665 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3c6fd-45fb-467b-816c-18b3e6cc9d4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:43.147998 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3c6fd-45fb-467b-816c-18b3e6cc9d4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:43.149433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3c6fd-45fb-467b-816c-18b3e6cc9d4e None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:43.153173 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:43.164173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 256/497] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:43.164173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3c6fd-45fb-467b-816c-18b3e6cc9d4e None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:43.177615 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7ecb7-8042-40ea-9c77-3c0acd5afe02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:43.178259 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7ecb7-8042-40ea-9c77-3c0acd5afe02 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:43.178549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7ecb7-8042-40ea-9c77-3c0acd5afe02 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:43.178825 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7ecb7-8042-40ea-9c77-3c0acd5afe02 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:43.179574 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7ecb7-8042-40ea-9c77-3c0acd5afe02 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:43.179883 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7ecb7-8042-40ea-9c77-3c0acd5afe02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:43.181270 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7ecb7-8042-40ea-9c77-3c0acd5afe02 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:43.189840 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 242/498] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jul 26 04:46:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:43.191074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e531a-8e07-4020-af6d-eba692e7d87b None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:43.191395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e531a-8e07-4020-af6d-eba692e7d87b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:43.191669 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e531a-8e07-4020-af6d-eba692e7d87b None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:43.193092 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 243/499] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:43.197576 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e531a-8e07-4020-af6d-eba692e7d87b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:43.216480 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:43.283075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3c6fd-45fb-467b-816c-18b3e6cc9d4e None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:43.306139 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e531a-8e07-4020-af6d-eba692e7d87b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:43.317546 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933474f-d520-4bb9-8f1b-a8e8c099c871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:43.318173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933474f-d520-4bb9-8f1b-a8e8c099c871 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:43.318480 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933474f-d520-4bb9-8f1b-a8e8c099c871 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:43.318730 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933474f-d520-4bb9-8f1b-a8e8c099c871 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:43.344512 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad6a4b-2887-46b6-b135-65b2c84c465b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:43.345178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad6a4b-2887-46b6-b135-65b2c84c465b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:43.345460 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad6a4b-2887-46b6-b135-65b2c84c465b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:43.345733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad6a4b-2887-46b6-b135-65b2c84c465b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:43.356210 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ad6a4b-2887-46b6-b135-65b2c84c465b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:43.358062 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ad6a4b-2887-46b6-b135-65b2c84c465b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:43.372721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a0ad6a4b-2887-46b6-b135-65b2c84c465b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 04:46:43.372721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:43.372721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:43.372721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:43.372721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:43.372721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:43.372721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:43.372721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:43.372721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:43.372721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:43.372721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:43.372721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:43.372721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:46:43.372721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:46:43.372721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:46:43.372721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:46:43.373845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:43.373845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:43.373845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:46:43.373845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:43.373845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:46:43.373845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:43.373845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:43.373845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:43.373845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:43.373845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:43.373845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:43.373845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:43.373845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:43.373845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:43.373845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:43.373845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:43.373845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:43.373845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:43.374720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:43.374720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:43.374720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:46:43.374720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:46:43.374720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:46:43.374720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:46:43.374720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:46:43.374720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:46:43.374720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 04:46:43.374720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:43.374720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 244/500] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:43.374720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad6a4b-2887-46b6-b135-65b2c84c465b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:43.414613 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933474f-d520-4bb9-8f1b-a8e8c099c871 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:43.416191 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933474f-d520-4bb9-8f1b-a8e8c099c871 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:43.429532 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:43.460816 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:43.493677 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad6a4b-2887-46b6-b135-65b2c84c465b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:43.526058 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 257/501] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:46:43.527429 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e780f8e-e2f0-4694-8d77-9de8deb5aba8 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:43.527816 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e780f8e-e2f0-4694-8d77-9de8deb5aba8 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:43.528134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e780f8e-e2f0-4694-8d77-9de8deb5aba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:43.538159 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e681880-c7bd-469a-9ffe-f48e04bba2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:43.538741 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e681880-c7bd-469a-9ffe-f48e04bba2cd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:43.539025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e681880-c7bd-469a-9ffe-f48e04bba2cd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:43.539282 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e681880-c7bd-469a-9ffe-f48e04bba2cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:43.546282 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-7e780f8e-e2f0-4694-8d77-9de8deb5aba8 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:43.547512 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-7e780f8e-e2f0-4694-8d77-9de8deb5aba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:43.551465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e681880-c7bd-469a-9ffe-f48e04bba2cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:43.552984 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e681880-c7bd-469a-9ffe-f48e04bba2cd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:43.570636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-1e681880-c7bd-469a-9ffe-f48e04bba2cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:43.570636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:43.570636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:43.570636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:43.570636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:43.570636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:43.570636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:43.570636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:43.570636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:43.570636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:43.570636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:43.570636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:43.570636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:43.570636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:43.570636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:43.571749 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:43.571749 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:43.571749 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:43.571749 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:43.571749 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:43.571749 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:43.571749 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:43.571749 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:43.571749 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:43.571749 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:43.571749 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:43.571749 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:43.571749 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:43.571749 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:43.571749 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:43.571749 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:43.571749 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:43.572733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:43.572733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:43.572733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:43.572733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:43.572733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 245/502] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:43.573008 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e681880-c7bd-469a-9ffe-f48e04bba2cd None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:43.625240 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 258/503] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:43.626113 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e780f8e-e2f0-4694-8d77-9de8deb5aba8 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:43.679411 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:43.692324 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e681880-c7bd-469a-9ffe-f48e04bba2cd None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:43.702263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:43.732381 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7001d-b04c-4800-a891-ecf6687f6ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:43.733043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7001d-b04c-4800-a891-ecf6687f6ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:43.733312 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7001d-b04c-4800-a891-ecf6687f6ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:43.733592 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7001d-b04c-4800-a891-ecf6687f6ab6 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:43.734402 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7001d-b04c-4800-a891-ecf6687f6ab6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:43.734712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7001d-b04c-4800-a891-ecf6687f6ab6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:43.736223 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7001d-b04c-4800-a891-ecf6687f6ab6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:43.741920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e780f8e-e2f0-4694-8d77-9de8deb5aba8 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:43.754938 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 246/504] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Jul 26 04:46:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:43.756079 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06f285-b096-4e32-b5ba-e78393441729 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:43.756348 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06f285-b096-4e32-b5ba-e78393441729 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:43.756623 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06f285-b096-4e32-b5ba-e78393441729 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:43.758102 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 247/505] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:43.759493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06f285-b096-4e32-b5ba-e78393441729 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:43.782082 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51789bc4-ebb1-4d52-ba4f-9c409b770457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:43.782742 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51789bc4-ebb1-4d52-ba4f-9c409b770457 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:43.783026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51789bc4-ebb1-4d52-ba4f-9c409b770457 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:43.783306 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51789bc4-ebb1-4d52-ba4f-9c409b770457 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:43.794565 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51789bc4-ebb1-4d52-ba4f-9c409b770457 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:43.796028 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51789bc4-ebb1-4d52-ba4f-9c409b770457 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:43.811685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-51789bc4-ebb1-4d52-ba4f-9c409b770457 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:43.811685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:43.811685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:43.811685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:43.811685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:43.811685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:43.811685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:43.811685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:43.811685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:43.811685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:43.811685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:43.811685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:43.811685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:43.811685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:43.811685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:43.812788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:43.812788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:43.812788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:43.812788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:43.812788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:43.812788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:43.812788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:43.812788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:43.812788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:43.812788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:43.812788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:43.812788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:43.812788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:43.812788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:43.812788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:43.812788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:43.812788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:43.813841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:43.813841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:43.813841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:43.813841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:43.813841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 259/506] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:43.814130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51789bc4-ebb1-4d52-ba4f-9c409b770457 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:43.876576 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06f285-b096-4e32-b5ba-e78393441729 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:43.911879 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c1bf5-6e51-4df9-aa6b-267234e6be3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:43.912517 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c1bf5-6e51-4df9-aa6b-267234e6be3d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:43.912795 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c1bf5-6e51-4df9-aa6b-267234e6be3d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:43.913070 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c1bf5-6e51-4df9-aa6b-267234e6be3d None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:43.913780 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c1bf5-6e51-4df9-aa6b-267234e6be3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:43.914079 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c1bf5-6e51-4df9-aa6b-267234e6be3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:43.915482 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c1bf5-6e51-4df9-aa6b-267234e6be3d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:43.927066 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 248/507] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:43.927930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c1bf5-6e51-4df9-aa6b-267234e6be3d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:43.929500 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51789bc4-ebb1-4d52-ba4f-9c409b770457 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:43.939827 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:43.952576 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:43.967880 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6050fd-91b8-48ac-80b6-90315591b9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:43.968534 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6050fd-91b8-48ac-80b6-90315591b9ab None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:43.968813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6050fd-91b8-48ac-80b6-90315591b9ab None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:43.969091 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6050fd-91b8-48ac-80b6-90315591b9ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:43.981013 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6050fd-91b8-48ac-80b6-90315591b9ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:43.986364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-fd6050fd-91b8-48ac-80b6-90315591b9ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:43.986364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:43.986364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:43.986364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:43.986364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:43.986364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:43.986364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:43.986364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:43.986364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:43.986364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:43.986364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:43.986364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:43.986364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:43.986364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:43.986364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:43.986364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:43.987380 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:43.987380 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:43.987380 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:43.987380 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:43.987380 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:43.987380 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:43.987380 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:43.987380 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:43.987380 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:43.987380 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:43.987380 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:43.987380 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:43.987380 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:43.987380 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:43.987380 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:43.987380 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:43.988339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:43.988339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:43.988339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:43.988339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:43.988339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:43.988339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:43.988339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:43.988339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:43.988339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:43.988339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:43.988339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:43.988339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:43.988339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 260/508] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jul 26 04:46:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:43.989205 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86375065-60fa-4b90-a76c-d9c3c1dd9b01 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:43.989205 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86375065-60fa-4b90-a76c-d9c3c1dd9b01 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:43.989342 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86375065-60fa-4b90-a76c-d9c3c1dd9b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:44.009235 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-86375065-60fa-4b90-a76c-d9c3c1dd9b01 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:44.010458 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-86375065-60fa-4b90-a76c-d9c3c1dd9b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:44.036572 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c1bf5-6e51-4df9-aa6b-267234e6be3d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:44.072267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffb0ce-2532-4516-bd8e-98d82f28529e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:44.072817 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffb0ce-2532-4516-bd8e-98d82f28529e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:44.073152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffb0ce-2532-4516-bd8e-98d82f28529e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:44.073366 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffb0ce-2532-4516-bd8e-98d82f28529e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:44.079977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 261/509] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:44.080818 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86375065-60fa-4b90-a76c-d9c3c1dd9b01 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:44.145122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffb0ce-2532-4516-bd8e-98d82f28529e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:44.146621 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffb0ce-2532-4516-bd8e-98d82f28529e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:44.157328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:44.203019 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86375065-60fa-4b90-a76c-d9c3c1dd9b01 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:44.221655 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:44.247966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067ea6a-d2b2-4051-a8f4-35fc24dcc295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:44.248590 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067ea6a-d2b2-4051-a8f4-35fc24dcc295 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:44.248869 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067ea6a-d2b2-4051-a8f4-35fc24dcc295 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:44.249123 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067ea6a-d2b2-4051-a8f4-35fc24dcc295 None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:44.249821 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d067ea6a-d2b2-4051-a8f4-35fc24dcc295 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:44.250117 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d067ea6a-d2b2-4051-a8f4-35fc24dcc295 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:44.251165 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 249/510] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:46:44.251512 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d067ea6a-d2b2-4051-a8f4-35fc24dcc295 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:44.252051 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffb0ce-2532-4516-bd8e-98d82f28529e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:44.263783 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 262/511] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:44.264596 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067ea6a-d2b2-4051-a8f4-35fc24dcc295 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:44.357209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffb0ce-2532-4516-bd8e-98d82f28529e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:44.367585 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:44.368844 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067ea6a-d2b2-4051-a8f4-35fc24dcc295 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:44.394664 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1c8de-0f2a-4f71-968c-2dcda9d5dcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:44.395322 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1c8de-0f2a-4f71-968c-2dcda9d5dcd2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:44.395619 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1c8de-0f2a-4f71-968c-2dcda9d5dcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:44.395915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1c8de-0f2a-4f71-968c-2dcda9d5dcd2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:44.403774 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d1c8de-0f2a-4f71-968c-2dcda9d5dcd2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:44.403925 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e89711c-b9d4-42f1-9de3-2c316b054e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:44.404562 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89711c-b9d4-42f1-9de3-2c316b054e61 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:44.404847 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89711c-b9d4-42f1-9de3-2c316b054e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:44.405122 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89711c-b9d4-42f1-9de3-2c316b054e61 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:44.405278 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d1c8de-0f2a-4f71-968c-2dcda9d5dcd2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:44.405835 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e89711c-b9d4-42f1-9de3-2c316b054e61 None admin] RBAC: Adding query filter params (name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:44.406148 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e89711c-b9d4-42f1-9de3-2c316b054e61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:44.407524 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e89711c-b9d4-42f1-9de3-2c316b054e61 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:44.416123 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 263/512] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jul 26 04:46:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:44.416957 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e89711c-b9d4-42f1-9de3-2c316b054e61 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:44.417392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e3d1c8de-0f2a-4f71-968c-2dcda9d5dcd2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 26 04:46:44.417392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:44.417392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:44.417392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:44.417392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:44.417392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:44.417392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:44.417392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:44.417392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:44.417392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:44.417392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:44.417392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:44.417392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:46:44.417392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:46:44.417392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:46:44.417392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:46:44.418342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:44.418342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:44.418342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:46:44.418342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:44.418342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:46:44.418342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:44.418342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:44.418342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:44.418342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:44.418342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:44.418342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:44.418342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:44.418342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:44.418342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:44.418342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:44.418342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:44.418342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:44.418342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:44.419327 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:44.419327 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:44.419327 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:46:44.419327 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:46:44.419327 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:46:44.419327 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:46:44.419327 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:46:44.419327 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:46:44.419327 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 26 04:46:44.419327 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:44.419327 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 250/513] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:44.420412 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1c8de-0f2a-4f71-968c-2dcda9d5dcd2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:44.455706 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:44.523124 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1c8de-0f2a-4f71-968c-2dcda9d5dcd2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:44.529510 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e89711c-b9d4-42f1-9de3-2c316b054e61 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:44.556832 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf386ce-236a-491b-b027-94d77d3555a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:44.557477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf386ce-236a-491b-b027-94d77d3555a3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:44.557760 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf386ce-236a-491b-b027-94d77d3555a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:44.558056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf386ce-236a-491b-b027-94d77d3555a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:44.563285 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2dd60e-3bfd-45bb-a314-8552f38aa755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:44.563958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dd60e-3bfd-45bb-a314-8552f38aa755 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:44.564237 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dd60e-3bfd-45bb-a314-8552f38aa755 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:44.564505 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dd60e-3bfd-45bb-a314-8552f38aa755 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:44.568529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf386ce-236a-491b-b027-94d77d3555a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:44.569999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf386ce-236a-491b-b027-94d77d3555a3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:44.575513 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2dd60e-3bfd-45bb-a314-8552f38aa755 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:44.576984 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2dd60e-3bfd-45bb-a314-8552f38aa755 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:44.584080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-edf386ce-236a-491b-b027-94d77d3555a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:44.584080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:44.584080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:44.584080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:44.584080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:44.584080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:44.584080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:44.584080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:44.584080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:44.584080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:44.584080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:44.584080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:44.584080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:44.584080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:44.584080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:44.584923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:44.584923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:44.584923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:44.584923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:44.584923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:44.584923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:44.584923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:44.584923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:44.584923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:44.584923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:44.584923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:44.584923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:44.584923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:44.584923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:44.584923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:44.584923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:44.584923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:44.585753 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:44.585753 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:44.585753 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:44.585753 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:44.585753 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 251/514] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:44.586015 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf386ce-236a-491b-b027-94d77d3555a3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:44.594579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-5d2dd60e-3bfd-45bb-a314-8552f38aa755 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:44.594579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:44.594579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:44.594579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:44.594579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:44.594579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:44.594579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:44.594579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:44.594579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:44.594579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:44.594579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:44.594579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:44.594579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:44.594579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:44.594579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:44.595455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:44.595455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:44.595455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:44.595455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:44.595455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:44.595455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:44.595455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:44.595455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:44.595455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:44.595455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:44.595455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:44.595455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:44.595455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:44.595455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:44.595455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:44.595455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:44.595455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:44.596351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:44.596351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:44.596351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:44.596351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:44.596351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 264/515] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:44.596608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2dd60e-3bfd-45bb-a314-8552f38aa755 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:44.610989 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:44.683570 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:44.692260 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf386ce-236a-491b-b027-94d77d3555a3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:44.700639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2dd60e-3bfd-45bb-a314-8552f38aa755 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:44.726986 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4ea80-03c3-45ce-b642-9cb8f3e6fc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:44.727672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4ea80-03c3-45ce-b642-9cb8f3e6fc5c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:44.727964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4ea80-03c3-45ce-b642-9cb8f3e6fc5c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:44.728239 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4ea80-03c3-45ce-b642-9cb8f3e6fc5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:44.735126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4ea80-03c3-45ce-b642-9cb8f3e6fc5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:44.735269 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520c35d-f22a-4dd9-9488-eb1eac4e731c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:44.735776 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520c35d-f22a-4dd9-9488-eb1eac4e731c None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:44.736057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520c35d-f22a-4dd9-9488-eb1eac4e731c None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:44.736331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520c35d-f22a-4dd9-9488-eb1eac4e731c None admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:44.736609 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4ea80-03c3-45ce-b642-9cb8f3e6fc5c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:44.737138 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c520c35d-f22a-4dd9-9488-eb1eac4e731c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:44.737445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c520c35d-f22a-4dd9-9488-eb1eac4e731c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:44.738904 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c520c35d-f22a-4dd9-9488-eb1eac4e731c None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:44.748856 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-9fc4ea80-03c3-45ce-b642-9cb8f3e6fc5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:46:44.748856 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:44.748856 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:44.748856 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:44.748856 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:44.748856 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:44.748856 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:44.748856 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:44.748856 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:44.748856 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:44.748856 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:44.748856 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:44.748856 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:46:44.748856 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:46:44.748856 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:46:44.749777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:46:44.749777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:44.749777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:44.749777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:44.749777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:44.749777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:44.749777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:44.749777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:44.749777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:44.749777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:44.749777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:44.749777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:44.749777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:44.749777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:44.749777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:44.749777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:44.749777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:44.750622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:46:44.750622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:46:44.750622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:46:44.750622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:46:44.750622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:46:44.750622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:46:44.750622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:46:44.750622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:44.750622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 252/516] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:46:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:44.751106 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4ea80-03c3-45ce-b642-9cb8f3e6fc5c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:44.777204 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 265/517] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Jul 26 04:46:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:44.778157 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520c35d-f22a-4dd9-9488-eb1eac4e731c None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:44.836510 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:44.860224 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4ea80-03c3-45ce-b642-9cb8f3e6fc5c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:44.886369 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520c35d-f22a-4dd9-9488-eb1eac4e731c None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:44.895617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eda34a-cfff-406c-a858-8f4e7bf71f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:44.896289 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eda34a-cfff-406c-a858-8f4e7bf71f0a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:44.896576 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eda34a-cfff-406c-a858-8f4e7bf71f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:44.896854 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eda34a-cfff-406c-a858-8f4e7bf71f0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:44.908534 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eda34a-cfff-406c-a858-8f4e7bf71f0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:44.910031 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eda34a-cfff-406c-a858-8f4e7bf71f0a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:44.922701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882004e-4ad4-494c-8d6d-6de516d125c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:44.923306 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882004e-4ad4-494c-8d6d-6de516d125c5 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:44.923614 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882004e-4ad4-494c-8d6d-6de516d125c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:44.923889 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882004e-4ad4-494c-8d6d-6de516d125c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:44.924581 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8882004e-4ad4-494c-8d6d-6de516d125c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:44.924752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-76eda34a-cfff-406c-a858-8f4e7bf71f0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:44.924752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:44.924752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:44.924752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:44.924752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:44.924752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:44.924752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:44.924752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:44.924752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:44.924752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:44.924752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:44.924752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:44.924752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:44.924752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:44.924752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:44.925704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:44.925704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:44.925704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:44.925704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:44.925704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:44.925704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:44.925704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:44.925704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:44.925704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:44.925704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:44.925704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:44.925704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:44.925704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:44.925704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:44.925704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:44.925704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:44.925704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:44.926598 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:44.926598 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:44.926598 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:44.926598 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:44.926807 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8882004e-4ad4-494c-8d6d-6de516d125c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:44.926881 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 253/518] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:44.926953 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8882004e-4ad4-494c-8d6d-6de516d125c5 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:44.927238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eda34a-cfff-406c-a858-8f4e7bf71f0a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:44.940992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 266/519] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:44.941844 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882004e-4ad4-494c-8d6d-6de516d125c5 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:44.944819 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:45.038732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eda34a-cfff-406c-a858-8f4e7bf71f0a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:45.043704 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882004e-4ad4-494c-8d6d-6de516d125c5 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:45.068730 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:45.074708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6da180-024e-453b-b76a-fdfac0e5e6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:45.075390 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6da180-024e-453b-b76a-fdfac0e5e6b7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:45.075705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6da180-024e-453b-b76a-fdfac0e5e6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:45.075977 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6da180-024e-453b-b76a-fdfac0e5e6b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:45.076678 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6da180-024e-453b-b76a-fdfac0e5e6b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:45.076971 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6da180-024e-453b-b76a-fdfac0e5e6b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:45.077780 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fa3e5-0cf8-43bb-9334-660e8f968fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:45.078356 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6da180-024e-453b-b76a-fdfac0e5e6b7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:45.078481 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fa3e5-0cf8-43bb-9334-660e8f968fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:45.078675 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fa3e5-0cf8-43bb-9334-660e8f968fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:45.078949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fa3e5-0cf8-43bb-9334-660e8f968fd5 None admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:45.079771 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102fa3e5-0cf8-43bb-9334-660e8f968fd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:45.080109 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102fa3e5-0cf8-43bb-9334-660e8f968fd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:45.081567 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102fa3e5-0cf8-43bb-9334-660e8f968fd5 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:45.090470 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 254/520] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:45.091168 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6da180-024e-453b-b76a-fdfac0e5e6b7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:45.100407 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 267/521] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:46:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:45.101291 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fa3e5-0cf8-43bb-9334-660e8f968fd5 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:45.165002 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:45.194006 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6da180-024e-453b-b76a-fdfac0e5e6b7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:45.204636 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fa3e5-0cf8-43bb-9334-660e8f968fd5 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:45.228771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144dd84-e468-45a9-9e90-0d71ae1f10eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:45.229420 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144dd84-e468-45a9-9e90-0d71ae1f10eb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:45.229734 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144dd84-e468-45a9-9e90-0d71ae1f10eb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:45.229977 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144dd84-e468-45a9-9e90-0d71ae1f10eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:45.238581 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b5ec5-350c-4320-b5f0-b6745ddfa76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:45.239228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b5ec5-350c-4320-b5f0-b6745ddfa76f None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:45.239502 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b5ec5-350c-4320-b5f0-b6745ddfa76f None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:45.239797 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b5ec5-350c-4320-b5f0-b6745ddfa76f None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:45.240499 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b5ec5-350c-4320-b5f0-b6745ddfa76f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:45.240798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b5ec5-350c-4320-b5f0-b6745ddfa76f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:45.242212 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b5ec5-350c-4320-b5f0-b6745ddfa76f None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:45.254032 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 268/522] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:45.254855 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b5ec5-350c-4320-b5f0-b6745ddfa76f None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:45.294886 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:45.305665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a144dd84-e468-45a9-9e90-0d71ae1f10eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:45.307154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a144dd84-e468-45a9-9e90-0d71ae1f10eb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:45.360374 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b5ec5-350c-4320-b5f0-b6745ddfa76f None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:45.382903 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:45.396246 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1157ac-e704-49b5-bb46-66cc6defc454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:45.396898 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1157ac-e704-49b5-bb46-66cc6defc454 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:45.397538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1157ac-e704-49b5-bb46-66cc6defc454 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:45.397538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1157ac-e704-49b5-bb46-66cc6defc454 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:45.404327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1157ac-e704-49b5-bb46-66cc6defc454 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:45.405814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1157ac-e704-49b5-bb46-66cc6defc454 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:45.411067 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 255/523] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:46:45.411944 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144dd84-e468-45a9-9e90-0d71ae1f10eb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:45.422639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-ee1157ac-e704-49b5-bb46-66cc6defc454 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:46:45.422639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:45.422639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:45.422639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:45.422639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:45.422639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:45.422639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:45.422639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:45.422639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:45.422639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:45.422639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:45.422639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:45.422639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:46:45.422639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:46:45.422639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:46:45.423959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:46:45.423959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:45.423959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:45.423959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:45.423959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:45.423959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:45.423959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:45.423959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:45.423959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:45.423959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:45.423959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:45.423959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:45.423959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:45.423959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:45.423959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:45.423959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:45.423959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:45.424825 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:46:45.424825 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:46:45.424825 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:46:45.424825 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:46:45.424825 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:46:45.424825 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:46:45.424825 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:46:45.424825 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:45.424825 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 269/524] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:46:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:45.425296 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1157ac-e704-49b5-bb46-66cc6defc454 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:45.506976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:45.516476 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144dd84-e468-45a9-9e90-0d71ae1f10eb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:45.530619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1157ac-e704-49b5-bb46-66cc6defc454 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:45.550663 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b2f90-6592-44fc-8bd1-7b21634e1938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:45.551298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b2f90-6592-44fc-8bd1-7b21634e1938 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:45.551598 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b2f90-6592-44fc-8bd1-7b21634e1938 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:45.551876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b2f90-6592-44fc-8bd1-7b21634e1938 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:45.552564 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b2f90-6592-44fc-8bd1-7b21634e1938 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:45.554011 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b2f90-6592-44fc-8bd1-7b21634e1938 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:45.564655 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f05ab-f35d-471b-8cc5-dd285b9e1d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:45.565275 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f05ab-f35d-471b-8cc5-dd285b9e1d26 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:45.565552 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f05ab-f35d-471b-8cc5-dd285b9e1d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:45.565815 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f05ab-f35d-471b-8cc5-dd285b9e1d26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:45.566616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f05ab-f35d-471b-8cc5-dd285b9e1d26 None admin] RBAC: Adding query filter params (role.id=e5048f4e7c6b4494a15d628758090380, scope.project.id=d583e0be8ef74766bffd6c3c07af18f1, user.id=2faeed9002ee4154bc980c146cd06bcc) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:45.566920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f05ab-f35d-471b-8cc5-dd285b9e1d26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:45.568610 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f05ab-f35d-471b-8cc5-dd285b9e1d26 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:45.574403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 270/525] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:46:45 2022] GET /identity/v3/role_assignments?user.id=2faeed9002ee4154bc980c146cd06bcc&scope.project.id=d583e0be8ef74766bffd6c3c07af18f1&role.id=e5048f4e7c6b4494a15d628758090380 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:45.575314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f05ab-f35d-471b-8cc5-dd285b9e1d26 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:45.575905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 256/526] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jul 26 04:46:45 2022] POST /identity/v3/users => generated 247 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:46:45.576813 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b2f90-6592-44fc-8bd1-7b21634e1938 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:45.704527 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:45.729540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b2f90-6592-44fc-8bd1-7b21634e1938 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:45.773399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f05ab-f35d-471b-8cc5-dd285b9e1d26 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:45.783152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0bc434-5da4-4460-a940-79f02e7a79b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:45.783891 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0bc434-5da4-4460-a940-79f02e7a79b6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:45.784558 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0bc434-5da4-4460-a940-79f02e7a79b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:45.784558 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0bc434-5da4-4460-a940-79f02e7a79b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:45.797366 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0bc434-5da4-4460-a940-79f02e7a79b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:45.798857 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0bc434-5da4-4460-a940-79f02e7a79b6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:45.807474 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:45.816189 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-2d0bc434-5da4-4460-a940-79f02e7a79b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:45.816189 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:45.816189 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:45.816189 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:45.816189 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:45.816189 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:45.816189 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:45.816189 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:45.816189 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:45.816189 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:45.816189 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:45.816189 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:45.816189 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:45.816189 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:45.816189 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:45.817129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:45.817129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:45.817129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:45.817129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:45.817129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:45.817129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:45.817129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:45.817129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:45.817129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:45.817129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:45.817129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:45.817129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:45.817129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:45.817129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:45.817129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:45.817129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:45.817129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:45.818005 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:45.818005 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:45.818005 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:45.818005 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:45.818005 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 257/527] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:45.818886 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0bc434-5da4-4460-a940-79f02e7a79b6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:45.823642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085974d0-424f-45e7-9854-eb90427d1012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:45.823642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085974d0-424f-45e7-9854-eb90427d1012 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:45.823642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085974d0-424f-45e7-9854-eb90427d1012 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:45.823642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085974d0-424f-45e7-9854-eb90427d1012 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:45.833116 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085974d0-424f-45e7-9854-eb90427d1012 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:45.834599 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085974d0-424f-45e7-9854-eb90427d1012 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:45.856439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-085974d0-424f-45e7-9854-eb90427d1012 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 04:46:45.856439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:45.856439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:45.856439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:45.856439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:45.856439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:45.856439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:45.856439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:45.856439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:45.856439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:45.856439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:45.856439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:45.856439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:46:45.856439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:46:45.856439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:46:45.856439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:46:45.857383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:45.857383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:45.857383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:46:45.857383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:45.857383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:46:45.857383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:45.857383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:45.857383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:45.857383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:45.857383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:45.857383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:45.857383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:45.857383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:45.857383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:45.857383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:45.857383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:45.857383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:45.857383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:45.858294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:45.858294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:45.858294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:46:45.858294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:46:45.858294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:46:45.858294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:46:45.858294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:46:45.858294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:46:45.858294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 04:46:45.858294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:45.858294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 271/528] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jul 26 04:46:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:45.858294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085974d0-424f-45e7-9854-eb90427d1012 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:45.963574 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0bc434-5da4-4460-a940-79f02e7a79b6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:45.968278 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085974d0-424f-45e7-9854-eb90427d1012 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:45.975762 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:46.007293 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f17b4c8-21bb-4a91-8995-9adc5eb879f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:46.008115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17b4c8-21bb-4a91-8995-9adc5eb879f1 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:46.008446 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17b4c8-21bb-4a91-8995-9adc5eb879f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:46.008717 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17b4c8-21bb-4a91-8995-9adc5eb879f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:46.009573 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f17b4c8-21bb-4a91-8995-9adc5eb879f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:46.009893 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f17b4c8-21bb-4a91-8995-9adc5eb879f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:46.011302 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f17b4c8-21bb-4a91-8995-9adc5eb879f1 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:46.011738 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38b205-3a1b-4dab-8735-3f12e56d3e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:46.012755 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38b205-3a1b-4dab-8735-3f12e56d3e91 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:46.013484 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38b205-3a1b-4dab-8735-3f12e56d3e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:46.014004 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38b205-3a1b-4dab-8735-3f12e56d3e91 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:46.015462 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b38b205-3a1b-4dab-8735-3f12e56d3e91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:46.015959 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b38b205-3a1b-4dab-8735-3f12e56d3e91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:46.017595 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b38b205-3a1b-4dab-8735-3f12e56d3e91 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:46.039639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 272/529] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:46.039639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f17b4c8-21bb-4a91-8995-9adc5eb879f1 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:46.048281 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 258/530] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:46:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:46.049300 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38b205-3a1b-4dab-8735-3f12e56d3e91 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:46.159986 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:46.166497 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f17b4c8-21bb-4a91-8995-9adc5eb879f1 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:46.204845 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37d347-b79c-41d8-af32-3d223c60c327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:46.205633 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37d347-b79c-41d8-af32-3d223c60c327 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:46.206012 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37d347-b79c-41d8-af32-3d223c60c327 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:46.206376 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37d347-b79c-41d8-af32-3d223c60c327 None admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:46.206902 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38b205-3a1b-4dab-8735-3f12e56d3e91 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:46.207756 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e37d347-b79c-41d8-af32-3d223c60c327 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:46.208155 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e37d347-b79c-41d8-af32-3d223c60c327 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:46.211025 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e37d347-b79c-41d8-af32-3d223c60c327 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:46.236657 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 273/531] 10.222.0.17 () {56 vars in 1139 bytes} [Tue Jul 26 04:46:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:46.237738 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37d347-b79c-41d8-af32-3d223c60c327 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:46.249362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a681f7-47d7-46bd-9d5c-d1c110cd2ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:46.250265 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a681f7-47d7-46bd-9d5c-d1c110cd2ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:46.250685 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a681f7-47d7-46bd-9d5c-d1c110cd2ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:46.251126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a681f7-47d7-46bd-9d5c-d1c110cd2ec7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:46.259357 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a681f7-47d7-46bd-9d5c-d1c110cd2ec7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:46.261226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a681f7-47d7-46bd-9d5c-d1c110cd2ec7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:46.269541 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:46.276425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-51a681f7-47d7-46bd-9d5c-d1c110cd2ec7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:46:46.276425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:46.276425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:46.276425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:46.276425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:46.276425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:46.276425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:46.276425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:46.276425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:46.276425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:46.276425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:46.276425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:46.276425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:46:46.276425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:46:46.276425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:46:46.277502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:46:46.277502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:46.277502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:46.277502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:46.277502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:46.277502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:46.277502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:46.277502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:46.277502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:46.277502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:46.277502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:46.277502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:46.277502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:46.277502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:46.277502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:46.277502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:46.277502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:46.278585 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:46:46.278585 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:46:46.278585 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:46:46.278585 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:46:46.278585 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:46:46.278585 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:46:46.278585 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:46:46.278585 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:46.286069 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 259/532] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:46:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:46.287240 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a681f7-47d7-46bd-9d5c-d1c110cd2ec7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:46.371068 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37d347-b79c-41d8-af32-3d223c60c327 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:46.417919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a681f7-47d7-46bd-9d5c-d1c110cd2ec7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:46.418526 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55774835-50ab-4a45-b81b-746a2cd45826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:46.419256 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55774835-50ab-4a45-b81b-746a2cd45826 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:46.421168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55774835-50ab-4a45-b81b-746a2cd45826 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:46.421168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55774835-50ab-4a45-b81b-746a2cd45826 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:46.422341 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55774835-50ab-4a45-b81b-746a2cd45826 None admin] RBAC: Adding query filter params (role.id=e5048f4e7c6b4494a15d628758090380, scope.project.id=d583e0be8ef74766bffd6c3c07af18f1, user.id=79ef3d83ac454fac9f890374cd22507a) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:46.427808 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55774835-50ab-4a45-b81b-746a2cd45826 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:46.430416 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55774835-50ab-4a45-b81b-746a2cd45826 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:46.447458 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 274/533] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:46:46 2022] GET /identity/v3/role_assignments?user.id=79ef3d83ac454fac9f890374cd22507a&scope.project.id=d583e0be8ef74766bffd6c3c07af18f1&role.id=e5048f4e7c6b4494a15d628758090380 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:46.447458 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55774835-50ab-4a45-b81b-746a2cd45826 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:46.455591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:46.488223 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824aa9ff-c445-4e38-8b19-7c4bde630ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:46.489231 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824aa9ff-c445-4e38-8b19-7c4bde630ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:46.489714 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824aa9ff-c445-4e38-8b19-7c4bde630ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:46.490182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824aa9ff-c445-4e38-8b19-7c4bde630ca9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:46.557533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:46.577030 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55774835-50ab-4a45-b81b-746a2cd45826 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:46.617353 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d45c2c1-5911-43b5-8b12-566e4437e7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:46.618448 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d45c2c1-5911-43b5-8b12-566e4437e7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:46.618828 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d45c2c1-5911-43b5-8b12-566e4437e7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:46.619197 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d45c2c1-5911-43b5-8b12-566e4437e7e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:46.620685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d45c2c1-5911-43b5-8b12-566e4437e7e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:46.621095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d45c2c1-5911-43b5-8b12-566e4437e7e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:46.623144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d45c2c1-5911-43b5-8b12-566e4437e7e2 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:46.639078 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 275/534] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:46:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:46.645248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824aa9ff-c445-4e38-8b19-7c4bde630ca9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:46.645773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d45c2c1-5911-43b5-8b12-566e4437e7e2 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:46.647446 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824aa9ff-c445-4e38-8b19-7c4bde630ca9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:46.786780 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d45c2c1-5911-43b5-8b12-566e4437e7e2 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:46.839168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc6737-e3bd-4b65-806d-ed5cf754ba5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:46.839963 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc6737-e3bd-4b65-806d-ed5cf754ba5f None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:46.840377 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc6737-e3bd-4b65-806d-ed5cf754ba5f None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:46.840725 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc6737-e3bd-4b65-806d-ed5cf754ba5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:46.841652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdc6737-e3bd-4b65-806d-ed5cf754ba5f None admin] RBAC: Adding query filter params (role.id=e5048f4e7c6b4494a15d628758090380, scope.project.id=d583e0be8ef74766bffd6c3c07af18f1, user.id=6946a788d2ef49ff873ca8e81990527e) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:46.842043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdc6737-e3bd-4b65-806d-ed5cf754ba5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:46.845135 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdc6737-e3bd-4b65-806d-ed5cf754ba5f None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:46.854244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 276/535] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:46:46 2022] GET /identity/v3/role_assignments?user.id=6946a788d2ef49ff873ca8e81990527e&scope.project.id=d583e0be8ef74766bffd6c3c07af18f1&role.id=e5048f4e7c6b4494a15d628758090380 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:46.876070 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 260/536] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:46:46.882374 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc6737-e3bd-4b65-806d-ed5cf754ba5f None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:46.910981 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:47.051508 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc6737-e3bd-4b65-806d-ed5cf754ba5f None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:47.093781 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2369819-0be4-41b9-9899-fcb01b5ffd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:47.094858 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2369819-0be4-41b9-9899-fcb01b5ffd2d None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:47.095341 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2369819-0be4-41b9-9899-fcb01b5ffd2d None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:47.095856 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2369819-0be4-41b9-9899-fcb01b5ffd2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:47.109796 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2369819-0be4-41b9-9899-fcb01b5ffd2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:47.111694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2369819-0be4-41b9-9899-fcb01b5ffd2d None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:47.129015 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-f2369819-0be4-41b9-9899-fcb01b5ffd2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:47.129015 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:47.129015 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:47.129015 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:47.129015 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:47.129015 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:47.129015 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:47.129015 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:47.129015 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:47.129015 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:47.129015 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:47.129015 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:47.129015 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:47.129015 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:47.129015 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:47.129915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:47.129915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:47.129915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:47.129915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:47.129915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:47.129915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:47.129915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:47.129915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:47.129915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:47.129915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:47.129915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:47.129915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:47.129915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:47.129915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:47.129915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:47.129915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:47.129915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:47.130733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:47.130733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:47.130733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:47.130733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:47.133085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 277/537] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:47.138988 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824aa9ff-c445-4e38-8b19-7c4bde630ca9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:47.166965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:47.269457 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824aa9ff-c445-4e38-8b19-7c4bde630ca9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:47.307003 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689ecfb-97b6-4eb7-aedd-d18136608ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:47.307584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689ecfb-97b6-4eb7-aedd-d18136608ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:47.307875 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689ecfb-97b6-4eb7-aedd-d18136608ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:47.308148 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689ecfb-97b6-4eb7-aedd-d18136608ce5 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:47.308918 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689ecfb-97b6-4eb7-aedd-d18136608ce5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:47.309226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689ecfb-97b6-4eb7-aedd-d18136608ce5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:47.310614 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689ecfb-97b6-4eb7-aedd-d18136608ce5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:47.327792 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 261/538] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:47.329411 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2369819-0be4-41b9-9899-fcb01b5ffd2d None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:47.422405 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:47.479602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2369819-0be4-41b9-9899-fcb01b5ffd2d None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:47.521955 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed51ca-da27-414c-8b9f-b4da11013f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:47.522672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed51ca-da27-414c-8b9f-b4da11013f1f None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:47.522957 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed51ca-da27-414c-8b9f-b4da11013f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:47.523232 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed51ca-da27-414c-8b9f-b4da11013f1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:47.531434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed51ca-da27-414c-8b9f-b4da11013f1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:47.533607 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed51ca-da27-414c-8b9f-b4da11013f1f None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:47.546594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-78ed51ca-da27-414c-8b9f-b4da11013f1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:46:47.546594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:47.546594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:47.546594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:47.546594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:47.546594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:47.546594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:47.546594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:47.546594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:47.546594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:47.546594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:47.546594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:47.546594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:46:47.546594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:46:47.546594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:46:47.547482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:46:47.547482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:47.547482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:47.547482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:47.547482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:47.547482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:47.547482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:47.547482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:47.547482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:47.547482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:47.547482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:47.547482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:47.547482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:47.547482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:47.547482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:47.547482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:47.547482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:47.548351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:46:47.548351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:46:47.548351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:46:47.548351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:46:47.548351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:46:47.548351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:46:47.548351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:46:47.548351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:47.548351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 278/539] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:46:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:47.554020 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689ecfb-97b6-4eb7-aedd-d18136608ce5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:47.640879 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:47.661548 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689ecfb-97b6-4eb7-aedd-d18136608ce5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:47.697378 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44651c-277d-4945-86b4-7eca85f1e8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:47.698030 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44651c-277d-4945-86b4-7eca85f1e8e6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:47.698315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44651c-277d-4945-86b4-7eca85f1e8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:47.698591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44651c-277d-4945-86b4-7eca85f1e8e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:47.699395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44651c-277d-4945-86b4-7eca85f1e8e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:47.699789 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44651c-277d-4945-86b4-7eca85f1e8e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:47.701403 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44651c-277d-4945-86b4-7eca85f1e8e6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:47.714657 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 262/540] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:46:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:47.721127 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed51ca-da27-414c-8b9f-b4da11013f1f None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:47.814889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c38e0-9284-4005-9e06-6886b27166e6 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:47.815239 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c38e0-9284-4005-9e06-6886b27166e6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:47.815431 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c38e0-9284-4005-9e06-6886b27166e6 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:47.816973 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 263/541] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:46:47.822360 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eec7d8-266d-4e59-b123-744f54b6a0ee None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:47.822643 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eec7d8-266d-4e59-b123-744f54b6a0ee None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:47.822917 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eec7d8-266d-4e59-b123-744f54b6a0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:47.850993 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d3eec7d8-266d-4e59-b123-744f54b6a0ee None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:47.852934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d3eec7d8-266d-4e59-b123-744f54b6a0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:47.864057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed51ca-da27-414c-8b9f-b4da11013f1f None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:47.895464 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:47.906883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf38f9-f7f8-47b8-b8f9-e510b3bbd61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:47.907640 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf38f9-f7f8-47b8-b8f9-e510b3bbd61a None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:47.908000 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf38f9-f7f8-47b8-b8f9-e510b3bbd61a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:47.908345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf38f9-f7f8-47b8-b8f9-e510b3bbd61a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:47.909236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf38f9-f7f8-47b8-b8f9-e510b3bbd61a None admin] RBAC: Adding query filter params (role.id=e5048f4e7c6b4494a15d628758090380, scope.project.id=d583e0be8ef74766bffd6c3c07af18f1, user.id=dbdb8997891241e9b2e829feb8023f8d) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:47.909623 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf38f9-f7f8-47b8-b8f9-e510b3bbd61a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:47.911176 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf38f9-f7f8-47b8-b8f9-e510b3bbd61a None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:47.917623 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 279/542] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:46:47 2022] GET /identity/v3/role_assignments?user.id=dbdb8997891241e9b2e829feb8023f8d&scope.project.id=d583e0be8ef74766bffd6c3c07af18f1&role.id=e5048f4e7c6b4494a15d628758090380 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:47.919057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f9bfe-95db-4aa8-b756-fca36270deee None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:47.919456 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f9bfe-95db-4aa8-b756-fca36270deee None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:47.919842 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f9bfe-95db-4aa8-b756-fca36270deee None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:47.925404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 280/543] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:47.927168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ba21b-8aab-441c-89fb-246b7c8e2650 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:47.927455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ba21b-8aab-441c-89fb-246b7c8e2650 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:47.927767 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ba21b-8aab-441c-89fb-246b7c8e2650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:47.946989 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-1b5ba21b-8aab-441c-89fb-246b7c8e2650 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:47.948268 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-1b5ba21b-8aab-441c-89fb-246b7c8e2650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:47.954423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 264/544] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 04:46:47.997437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eec7d8-266d-4e59-b123-744f54b6a0ee None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:48.089343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 281/545] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 04:46:48.116054 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:48.129488 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ba21b-8aab-441c-89fb-246b7c8e2650 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:48.134490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eec7d8-266d-4e59-b123-744f54b6a0ee None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:48.204385 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b686cad-4f68-473a-a060-2f479050d058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:48.205350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b686cad-4f68-473a-a060-2f479050d058 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:48.205816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b686cad-4f68-473a-a060-2f479050d058 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:48.206303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b686cad-4f68-473a-a060-2f479050d058 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:48.225020 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b686cad-4f68-473a-a060-2f479050d058 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:48.234400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-7b686cad-4f68-473a-a060-2f479050d058 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:48.234400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:48.234400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:48.234400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:48.234400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:48.234400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:48.234400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:48.234400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:48.234400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:48.234400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:48.234400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:48.234400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:48.234400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:48.234400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:48.234400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:48.234400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:48.235342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:48.235342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:48.235342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:48.235342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:48.235342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:48.235342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:48.235342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:48.235342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:48.235342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:48.235342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:48.235342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:48.235342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:48.235342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:48.235342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:48.235342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:48.235342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:48.236145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:48.236145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:48.236145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:48.236145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:48.236145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:48.236145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:48.236145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:48.236145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:48.236145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:48.236145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:48.236145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:48.236145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:48.237216 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 265/546] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jul 26 04:46:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:46:48.245555 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b686cad-4f68-473a-a060-2f479050d058 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:48.331316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ba21b-8aab-441c-89fb-246b7c8e2650 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:48.379984 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:48.408729 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b686cad-4f68-473a-a060-2f479050d058 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:48.413250 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e9e5a-18fa-49c4-bfb5-bdd018f46500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:48.413923 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e9e5a-18fa-49c4-bfb5-bdd018f46500 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:48.414211 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e9e5a-18fa-49c4-bfb5-bdd018f46500 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:48.414490 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e9e5a-18fa-49c4-bfb5-bdd018f46500 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:48.430419 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e9e5a-18fa-49c4-bfb5-bdd018f46500 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:48.437715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-3e2e9e5a-18fa-49c4-bfb5-bdd018f46500 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:48.437715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:48.437715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:48.437715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:48.437715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:48.437715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:48.437715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:48.437715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:48.437715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:48.437715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:48.437715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:48.437715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:48.437715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:48.437715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:48.437715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:48.437715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:48.438618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:48.438618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:48.438618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:48.438618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:48.438618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:48.438618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:48.438618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:48.438618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:48.438618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:48.438618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:48.438618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:48.438618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:48.438618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:48.438618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:48.438618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:48.438618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:48.439407 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:48.439407 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:48.439407 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:48.439407 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:48.439407 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:48.439407 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:48.439407 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:48.439407 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:48.439407 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:48.439407 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:48.439407 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:48.439407 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:48.439407 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 282/547] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jul 26 04:46:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:48.440095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:48.445782 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e9e5a-18fa-49c4-bfb5-bdd018f46500 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:48.446115 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c1f25-a08d-485a-b758-eeb430b0ff99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:48.446849 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c1f25-a08d-485a-b758-eeb430b0ff99 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:48.447195 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c1f25-a08d-485a-b758-eeb430b0ff99 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:48.447537 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c1f25-a08d-485a-b758-eeb430b0ff99 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:48.448363 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c1f25-a08d-485a-b758-eeb430b0ff99 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:48.448677 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c1f25-a08d-485a-b758-eeb430b0ff99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:48.450358 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c1f25-a08d-485a-b758-eeb430b0ff99 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:48.460825 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 266/548] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jul 26 04:46:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:48.466606 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c1f25-a08d-485a-b758-eeb430b0ff99 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:48.573440 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e9e5a-18fa-49c4-bfb5-bdd018f46500 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:48.649298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd69897-084b-4b66-b89f-efb847619019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:48.649944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd69897-084b-4b66-b89f-efb847619019 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:48.650231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd69897-084b-4b66-b89f-efb847619019 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:48.650507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd69897-084b-4b66-b89f-efb847619019 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:48.651247 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd69897-084b-4b66-b89f-efb847619019 None admin] RBAC: Adding query filter params (name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:48.651576 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd69897-084b-4b66-b89f-efb847619019 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:48.652909 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c1f25-a08d-485a-b758-eeb430b0ff99 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:48.653095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd69897-084b-4b66-b89f-efb847619019 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:48.670671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 283/549] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jul 26 04:46:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:48.676460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd69897-084b-4b66-b89f-efb847619019 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:48.699861 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fba6a-949b-46ff-a791-bde190c26463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:48.700565 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fba6a-949b-46ff-a791-bde190c26463 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:48.700798 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fba6a-949b-46ff-a791-bde190c26463 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:48.703573 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fba6a-949b-46ff-a791-bde190c26463 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:48.724647 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:48.725133 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1fba6a-949b-46ff-a791-bde190c26463 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:48.727912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1fba6a-949b-46ff-a791-bde190c26463 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:48.761528 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-3c1fba6a-949b-46ff-a791-bde190c26463 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:48.761528 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:48.761528 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:48.761528 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:48.761528 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:48.761528 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:48.761528 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:48.761528 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:48.761528 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:48.761528 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:48.761528 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:48.761528 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:48.761528 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:48.761528 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:48.761528 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:48.764298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:48.764298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:48.764298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:48.764298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:48.764298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:48.764298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:48.764298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:48.764298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:48.764298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:48.764298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:48.764298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:48.764298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:48.764298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:48.764298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:48.764298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:48.764298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:48.764298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:48.766427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:48.766427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:48.766427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:48.766427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:48.766427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 267/550] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:46:48.771574 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fba6a-949b-46ff-a791-bde190c26463 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:48.784132 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd69897-084b-4b66-b89f-efb847619019 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:48.819179 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274caeb-22b8-45b6-957c-286273955717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:48.819880 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274caeb-22b8-45b6-957c-286273955717 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:48.820166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274caeb-22b8-45b6-957c-286273955717 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:48.820444 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274caeb-22b8-45b6-957c-286273955717 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:48.832385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274caeb-22b8-45b6-957c-286273955717 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:48.833910 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274caeb-22b8-45b6-957c-286273955717 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:48.850186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-8274caeb-22b8-45b6-957c-286273955717 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:48.850186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:48.850186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:48.850186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:48.850186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:48.850186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:48.850186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:48.850186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:48.850186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:48.850186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:48.850186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:48.850186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:48.850186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:48.850186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:48.850186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:48.852313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:48.852313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:48.852313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:48.852313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:48.852313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:48.852313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:48.852313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:48.852313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:48.852313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:48.852313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:48.852313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:48.852313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:48.852313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:48.852313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:48.852313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:48.852313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:48.852313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:48.854524 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:48.854524 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:48.854524 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:48.854524 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:48.859654 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 284/551] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:46:48.859654 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274caeb-22b8-45b6-957c-286273955717 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:48.862369 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:48.957174 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:48.968376 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274caeb-22b8-45b6-957c-286273955717 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:48.990951 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fba6a-949b-46ff-a791-bde190c26463 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:49.010566 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0776dc9-084d-46d2-bcc2-67c154525bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:49.011287 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0776dc9-084d-46d2-bcc2-67c154525bdc None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:49.011591 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0776dc9-084d-46d2-bcc2-67c154525bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:49.011929 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0776dc9-084d-46d2-bcc2-67c154525bdc None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:49.012647 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0776dc9-084d-46d2-bcc2-67c154525bdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:49.012956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0776dc9-084d-46d2-bcc2-67c154525bdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:49.014370 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0776dc9-084d-46d2-bcc2-67c154525bdc None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:49.041753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 285/552] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:49.051993 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab3dee1-9dec-4d1c-8c26-ddd5214a4aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:49.052636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3dee1-9dec-4d1c-8c26-ddd5214a4aaf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:49.052922 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3dee1-9dec-4d1c-8c26-ddd5214a4aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:49.053202 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3dee1-9dec-4d1c-8c26-ddd5214a4aaf None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:49.053912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab3dee1-9dec-4d1c-8c26-ddd5214a4aaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:49.054220 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab3dee1-9dec-4d1c-8c26-ddd5214a4aaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:49.055642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab3dee1-9dec-4d1c-8c26-ddd5214a4aaf None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:49.058999 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0776dc9-084d-46d2-bcc2-67c154525bdc None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:49.075098 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 268/553] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:49.080205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab3dee1-9dec-4d1c-8c26-ddd5214a4aaf None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:49.168157 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0776dc9-084d-46d2-bcc2-67c154525bdc None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:49.206284 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97190d3b-23b5-4a17-8ed5-df46ae7c94bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:49.207038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97190d3b-23b5-4a17-8ed5-df46ae7c94bd None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:49.207378 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97190d3b-23b5-4a17-8ed5-df46ae7c94bd None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:49.207683 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97190d3b-23b5-4a17-8ed5-df46ae7c94bd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:49.215967 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97190d3b-23b5-4a17-8ed5-df46ae7c94bd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:49.217835 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97190d3b-23b5-4a17-8ed5-df46ae7c94bd None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:49.231236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-97190d3b-23b5-4a17-8ed5-df46ae7c94bd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 04:46:49.231236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:49.231236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:49.231236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:49.231236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:49.231236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:49.231236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:49.231236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:49.231236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:49.231236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:49.231236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:49.231236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:49.231236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:46:49.231236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:46:49.231236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:46:49.231236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:46:49.232326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:49.232326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:49.232326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:46:49.232326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:49.232326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:46:49.232326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:49.232326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:49.232326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:49.232326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:49.232326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:49.232326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:49.232326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:49.232326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:49.232326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:49.232326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:49.232326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:49.232326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:49.232326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:49.233231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:49.233231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:49.233231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:46:49.233231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:46:49.233231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:46:49.233231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:46:49.233231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:46:49.233231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:46:49.233231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 04:46:49.233231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:49.233231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 286/554] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Jul 26 04:46:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:46:49.238122 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97190d3b-23b5-4a17-8ed5-df46ae7c94bd None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:49.255325 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:49.286135 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab3dee1-9dec-4d1c-8c26-ddd5214a4aaf None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:49.288861 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:49.328280 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f11e1-4c6c-423e-b904-fc1b0cb54a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:49.328956 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f11e1-4c6c-423e-b904-fc1b0cb54a68 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:49.329239 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f11e1-4c6c-423e-b904-fc1b0cb54a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:49.329512 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f11e1-4c6c-423e-b904-fc1b0cb54a68 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:49.347605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f11e1-4c6c-423e-b904-fc1b0cb54a68 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:49.347605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f11e1-4c6c-423e-b904-fc1b0cb54a68 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:49.373158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-eb2f11e1-4c6c-423e-b904-fc1b0cb54a68 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 04:46:49.373158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:49.373158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:49.373158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:49.373158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:49.373158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:49.373158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:49.373158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:49.373158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:49.373158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:49.373158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:49.373158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:49.373158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:46:49.373158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:46:49.373158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:46:49.373158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:46:49.374154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:49.374154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:49.374154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:46:49.374154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:49.374154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:46:49.374154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:49.374154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:49.374154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:49.374154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:49.374154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:49.374154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:49.374154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:49.374154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:49.374154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:49.374154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:49.374154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:49.374154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:49.374154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:49.376104 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:49.376104 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:49.376104 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:46:49.376104 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:46:49.376104 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:46:49.376104 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:46:49.376104 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:46:49.376104 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:46:49.376104 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 04:46:49.376104 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:49.376104 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 269/555] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:46:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:46:49.384488 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f11e1-4c6c-423e-b904-fc1b0cb54a68 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:49.448712 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97190d3b-23b5-4a17-8ed5-df46ae7c94bd None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:49.507178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f11e1-4c6c-423e-b904-fc1b0cb54a68 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:49.513558 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26596304-e9b6-43f0-a383-bbaa093be3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:49.514332 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26596304-e9b6-43f0-a383-bbaa093be3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:49.514718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26596304-e9b6-43f0-a383-bbaa093be3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:49.515094 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26596304-e9b6-43f0-a383-bbaa093be3d1 None admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:49.516040 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26596304-e9b6-43f0-a383-bbaa093be3d1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:49.517051 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26596304-e9b6-43f0-a383-bbaa093be3d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:49.519129 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26596304-e9b6-43f0-a383-bbaa093be3d1 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:49.544801 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 287/556] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Jul 26 04:46:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:49.546403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4126d-9d1c-4a0c-aed5-080d6d85aca8 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:49.546693 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4126d-9d1c-4a0c-aed5-080d6d85aca8 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:49.546967 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4126d-9d1c-4a0c-aed5-080d6d85aca8 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:49.552333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 288/557] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:49.552333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028e63b-b474-4c44-a4db-bb66e0794232 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:49.552333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028e63b-b474-4c44-a4db-bb66e0794232 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:49.552333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028e63b-b474-4c44-a4db-bb66e0794232 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:49.552333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 289/558] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:49.552333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028e63b-b474-4c44-a4db-bb66e0794232 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:49.556108 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d5326-3c4c-46a3-9d53-4be44a9243b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:49.557009 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d5326-3c4c-46a3-9d53-4be44a9243b5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:49.557291 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d5326-3c4c-46a3-9d53-4be44a9243b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:49.558542 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d5326-3c4c-46a3-9d53-4be44a9243b5 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:49.559473 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d5326-3c4c-46a3-9d53-4be44a9243b5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:49.559806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d5326-3c4c-46a3-9d53-4be44a9243b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:49.561287 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d5326-3c4c-46a3-9d53-4be44a9243b5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:49.582120 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 270/559] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Jul 26 04:46:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:49.583411 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e01056-6570-4d8c-a334-66e08341eef9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:49.583733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e01056-6570-4d8c-a334-66e08341eef9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:49.584009 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e01056-6570-4d8c-a334-66e08341eef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:49.603822 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-63e01056-6570-4d8c-a334-66e08341eef9 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:49.605167 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-63e01056-6570-4d8c-a334-66e08341eef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:49.624086 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:49.645946 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:49.682062 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028e63b-b474-4c44-a4db-bb66e0794232 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:49.703586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 271/560] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:49.703586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51254a37-81be-4f39-9dfe-2576871954e8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:49.703586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51254a37-81be-4f39-9dfe-2576871954e8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:49.703586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51254a37-81be-4f39-9dfe-2576871954e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:49.719205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-51254a37-81be-4f39-9dfe-2576871954e8 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:49.720588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-51254a37-81be-4f39-9dfe-2576871954e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:49.735995 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476f6fd-e12e-4d46-882a-402ea3996235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:49.736668 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476f6fd-e12e-4d46-882a-402ea3996235 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:49.737108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476f6fd-e12e-4d46-882a-402ea3996235 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:49.737247 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476f6fd-e12e-4d46-882a-402ea3996235 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:49.749695 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476f6fd-e12e-4d46-882a-402ea3996235 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:49.751242 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476f6fd-e12e-4d46-882a-402ea3996235 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:49.769712 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-4476f6fd-e12e-4d46-882a-402ea3996235 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:49.769712 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:49.769712 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:49.769712 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:49.769712 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:49.769712 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:49.769712 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:49.769712 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:49.769712 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:49.769712 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:49.769712 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:49.769712 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:49.769712 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:49.769712 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:49.769712 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:49.770710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:49.770710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:49.770710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:49.770710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:49.770710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:49.770710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:49.770710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:49.770710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:49.770710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:49.770710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:49.770710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:49.770710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:49.770710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:49.770710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:49.770710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:49.770710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:49.770710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:49.771661 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:49.771661 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:49.771661 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:49.771661 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:49.771661 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 290/561] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:49.772470 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476f6fd-e12e-4d46-882a-402ea3996235 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:49.811606 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 272/562] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:49.811606 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51254a37-81be-4f39-9dfe-2576871954e8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:49.902095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476f6fd-e12e-4d46-882a-402ea3996235 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:49.910218 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:49.931341 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51254a37-81be-4f39-9dfe-2576871954e8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:49.932435 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:49.948157 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bb52f-f0fc-42f3-af2e-c9827a4b1cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:49.949005 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bb52f-f0fc-42f3-af2e-c9827a4b1cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:49.949619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bb52f-f0fc-42f3-af2e-c9827a4b1cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:49.950066 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bb52f-f0fc-42f3-af2e-c9827a4b1cd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:49.970603 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dfe258-0eac-4365-bea7-873aa004c1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:49.971261 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfe258-0eac-4365-bea7-873aa004c1c7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:49.971579 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfe258-0eac-4365-bea7-873aa004c1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:49.971849 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfe258-0eac-4365-bea7-873aa004c1c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:49.983340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dfe258-0eac-4365-bea7-873aa004c1c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:49.988283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-20dfe258-0eac-4365-bea7-873aa004c1c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:49.988283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:49.988283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:49.988283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:49.988283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:49.988283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:49.988283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:49.988283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:49.988283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:49.988283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:49.988283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:49.988283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:49.988283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:49.988283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:49.988283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:49.988283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:49.989382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:49.989382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:49.989382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:49.989382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:49.989382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:49.989382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:49.989382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:49.989382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:49.989382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:49.989382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:49.989382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:49.989382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:49.989382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:49.989382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:49.989382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:49.989382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:49.990188 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:49.990188 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:49.990188 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:49.990188 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:49.990188 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:49.990188 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:49.990188 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:49.990188 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:49.990188 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:49.990188 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:49.990188 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:49.990188 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:49.990188 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 273/563] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jul 26 04:46:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:49.990867 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dfe258-0eac-4365-bea7-873aa004c1c7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:50.037091 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331bb52f-f0fc-42f3-af2e-c9827a4b1cd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:50.038523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331bb52f-f0fc-42f3-af2e-c9827a4b1cd8 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:50.106011 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dfe258-0eac-4365-bea7-873aa004c1c7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:50.145744 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658b34f-47c9-40dc-b65c-b31df0b025b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:50.146387 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658b34f-47c9-40dc-b65c-b31df0b025b7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:50.146709 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658b34f-47c9-40dc-b65c-b31df0b025b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:50.146993 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658b34f-47c9-40dc-b65c-b31df0b025b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:50.147755 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e658b34f-47c9-40dc-b65c-b31df0b025b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:50.148170 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e658b34f-47c9-40dc-b65c-b31df0b025b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:50.149670 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e658b34f-47c9-40dc-b65c-b31df0b025b7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:50.162443 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 291/564] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:46:50.162443 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bb52f-f0fc-42f3-af2e-c9827a4b1cd8 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:50.167484 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 274/565] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:50.172169 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658b34f-47c9-40dc-b65c-b31df0b025b7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:50.175074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:50.192350 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:50.269687 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bb52f-f0fc-42f3-af2e-c9827a4b1cd8 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:50.277573 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658b34f-47c9-40dc-b65c-b31df0b025b7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:50.303539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450326d-5eed-4681-8a95-ba87dc772ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:50.304127 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450326d-5eed-4681-8a95-ba87dc772ddf None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:50.304407 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450326d-5eed-4681-8a95-ba87dc772ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:50.304679 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450326d-5eed-4681-8a95-ba87dc772ddf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:50.312212 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8706779-fea1-4953-ad8c-7a767ba1765f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:50.312836 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8706779-fea1-4953-ad8c-7a767ba1765f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:50.313115 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8706779-fea1-4953-ad8c-7a767ba1765f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:50.313387 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8706779-fea1-4953-ad8c-7a767ba1765f None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:50.314105 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8706779-fea1-4953-ad8c-7a767ba1765f None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:50.314407 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8706779-fea1-4953-ad8c-7a767ba1765f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:50.314657 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d450326d-5eed-4681-8a95-ba87dc772ddf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:50.315804 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8706779-fea1-4953-ad8c-7a767ba1765f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:50.319532 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-d450326d-5eed-4681-8a95-ba87dc772ddf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:50.319532 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:50.319532 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:50.319532 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:50.319532 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:50.319532 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:50.319532 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:50.319532 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:50.319532 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:50.319532 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:50.319532 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:50.319532 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:50.319532 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:50.319532 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:50.319532 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:50.319532 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:50.320444 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:50.320444 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:50.320444 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:50.320444 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:50.320444 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:50.320444 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:50.320444 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:50.320444 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:50.320444 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:50.320444 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:50.320444 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:50.320444 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:50.320444 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:50.320444 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:50.320444 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:50.320444 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:50.321271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:50.321271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:50.321271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:50.321271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:50.321271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:50.321271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:50.321271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:50.321271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:50.321271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:50.321271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:50.321271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:50.321271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:50.321271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 292/566] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jul 26 04:46:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:50.321948 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450326d-5eed-4681-8a95-ba87dc772ddf None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:50.327759 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 275/567] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jul 26 04:46:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:50.327759 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8706779-fea1-4953-ad8c-7a767ba1765f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:50.397433 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:50.422944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:50.438418 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8706779-fea1-4953-ad8c-7a767ba1765f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:50.441460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450326d-5eed-4681-8a95-ba87dc772ddf None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:50.478583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e15b32-ee87-4564-a09a-0e5d25b6f813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:50.479035 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403a864-c6d8-4911-a4ac-1c8aede88c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:50.479216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e15b32-ee87-4564-a09a-0e5d25b6f813 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:50.479494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e15b32-ee87-4564-a09a-0e5d25b6f813 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:50.479684 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403a864-c6d8-4911-a4ac-1c8aede88c3a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:50.479781 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e15b32-ee87-4564-a09a-0e5d25b6f813 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:50.479923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403a864-c6d8-4911-a4ac-1c8aede88c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:50.480210 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403a864-c6d8-4911-a4ac-1c8aede88c3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:50.488362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4403a864-c6d8-4911-a4ac-1c8aede88c3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:50.489863 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4403a864-c6d8-4911-a4ac-1c8aede88c3a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:50.490874 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e15b32-ee87-4564-a09a-0e5d25b6f813 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:50.492377 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e15b32-ee87-4564-a09a-0e5d25b6f813 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:50.503015 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-4403a864-c6d8-4911-a4ac-1c8aede88c3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:46:50.503015 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:50.503015 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:50.503015 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:50.503015 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:50.503015 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:50.503015 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:50.503015 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:50.503015 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:50.503015 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:50.503015 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:50.503015 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:50.503015 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:46:50.503015 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:46:50.503015 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:46:50.503999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:46:50.503999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:50.503999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:50.503999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:50.503999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:50.503999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:50.503999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:50.503999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:50.503999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:50.503999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:50.503999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:50.503999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:50.503999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:50.503999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:50.503999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:50.503999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:50.503999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:50.504871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:46:50.504871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:46:50.504871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:46:50.504871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:46:50.504871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:46:50.504871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:46:50.504871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:46:50.504871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:50.504871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 276/568] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:46:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:50.505834 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3790f0f-8bda-4a3e-96d5-1a56cce9c413 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:50.505834 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3790f0f-8bda-4a3e-96d5-1a56cce9c413 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:50.505988 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3790f0f-8bda-4a3e-96d5-1a56cce9c413 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:50.507653 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 277/569] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:50.507960 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3790f0f-8bda-4a3e-96d5-1a56cce9c413 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:50.509404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-c3e15b32-ee87-4564-a09a-0e5d25b6f813 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:50.509404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:50.509404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:50.509404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:50.509404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:50.509404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:50.509404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:50.509404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:50.509404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:50.509404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:50.509404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:50.509404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:50.509404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:50.509404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:50.509404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:50.510314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:50.510314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:50.510314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:50.510314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:50.510314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:50.510314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:50.510314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:50.510314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:50.510314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:50.510314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:50.510314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:50.510314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:50.510314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:50.510314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:50.510314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:50.510314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:50.510314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:50.512273 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:50.512273 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:50.512273 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:50.512273 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:50.512273 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 293/570] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:50.512273 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e15b32-ee87-4564-a09a-0e5d25b6f813 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:50.613112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e15b32-ee87-4564-a09a-0e5d25b6f813 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:50.615462 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3790f0f-8bda-4a3e-96d5-1a56cce9c413 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:50.638509 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:50.648548 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e1e62f-7d1a-4c63-b1ad-d89118372d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:50.649183 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1e62f-7d1a-4c63-b1ad-d89118372d0c None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:50.649477 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1e62f-7d1a-4c63-b1ad-d89118372d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:50.649750 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1e62f-7d1a-4c63-b1ad-d89118372d0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:50.649908 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc351b2-28cd-4b2e-8144-b5c2b85d4225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:50.650533 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc351b2-28cd-4b2e-8144-b5c2b85d4225 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:50.650811 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc351b2-28cd-4b2e-8144-b5c2b85d4225 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:50.651081 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc351b2-28cd-4b2e-8144-b5c2b85d4225 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:50.651691 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:50.651815 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc351b2-28cd-4b2e-8144-b5c2b85d4225 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:50.652110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc351b2-28cd-4b2e-8144-b5c2b85d4225 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:50.653602 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc351b2-28cd-4b2e-8144-b5c2b85d4225 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:50.661432 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e1e62f-7d1a-4c63-b1ad-d89118372d0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:50.661941 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 278/571] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jul 26 04:46:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:50.662751 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc351b2-28cd-4b2e-8144-b5c2b85d4225 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:50.662928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e1e62f-7d1a-4c63-b1ad-d89118372d0c None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:50.680166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-a3e1e62f-7d1a-4c63-b1ad-d89118372d0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:50.680166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:50.680166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:50.680166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:50.680166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:50.680166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:50.680166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:50.680166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:50.680166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:50.680166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:50.680166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:50.680166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:50.680166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:50.680166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:50.680166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:50.681169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:50.681169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:50.681169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:50.681169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:50.681169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:50.681169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:50.681169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:50.681169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:50.681169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:50.681169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:50.681169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:50.681169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:50.681169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:50.681169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:50.681169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:50.681169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:50.681169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:50.682053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:50.682053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:50.682053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:50.682053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:50.682053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 294/572] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:50.682567 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e3e9d-a73a-4515-aba8-2c485b356c5f None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:50.682851 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e3e9d-a73a-4515-aba8-2c485b356c5f None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:50.683127 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e3e9d-a73a-4515-aba8-2c485b356c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:50.701535 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-5c7e3e9d-a73a-4515-aba8-2c485b356c5f None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:50.702790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-5c7e3e9d-a73a-4515-aba8-2c485b356c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:50.768812 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc351b2-28cd-4b2e-8144-b5c2b85d4225 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:50.771821 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 295/573] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:50.772554 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e3e9d-a73a-4515-aba8-2c485b356c5f None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:50.817454 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add35fe6-bde1-455a-a634-a005e57cc3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:50.818118 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add35fe6-bde1-455a-a634-a005e57cc3ab None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:50.818403 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add35fe6-bde1-455a-a634-a005e57cc3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:50.818682 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add35fe6-bde1-455a-a634-a005e57cc3ab None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:50.819460 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add35fe6-bde1-455a-a634-a005e57cc3ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:50.819801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add35fe6-bde1-455a-a634-a005e57cc3ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:50.821325 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add35fe6-bde1-455a-a634-a005e57cc3ab None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:50.834768 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 279/574] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:46:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:50.835659 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add35fe6-bde1-455a-a634-a005e57cc3ab None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:50.887013 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e3e9d-a73a-4515-aba8-2c485b356c5f None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:50.887636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:50.905463 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:50.928570 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd75ccb-1086-486c-8bd7-27d91a3418ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:50.929238 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd75ccb-1086-486c-8bd7-27d91a3418ae None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:50.929510 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd75ccb-1086-486c-8bd7-27d91a3418ae None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:50.929753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd75ccb-1086-486c-8bd7-27d91a3418ae None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:50.930478 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd75ccb-1086-486c-8bd7-27d91a3418ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:50.930781 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd75ccb-1086-486c-8bd7-27d91a3418ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:50.932172 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd75ccb-1086-486c-8bd7-27d91a3418ae None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:50.943982 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 296/575] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:50.944872 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd75ccb-1086-486c-8bd7-27d91a3418ae None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:50.952674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add35fe6-bde1-455a-a634-a005e57cc3ab None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:50.987829 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f303a5-0440-4fdc-bb56-b7bf2243cff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:50.988484 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f303a5-0440-4fdc-bb56-b7bf2243cff3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:50.988761 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f303a5-0440-4fdc-bb56-b7bf2243cff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:50.989030 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f303a5-0440-4fdc-bb56-b7bf2243cff3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:50.999030 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f303a5-0440-4fdc-bb56-b7bf2243cff3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:51.000510 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f303a5-0440-4fdc-bb56-b7bf2243cff3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:51.015668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b0f303a5-0440-4fdc-bb56-b7bf2243cff3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:51.015668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:51.015668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:51.015668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:51.015668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:51.015668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:51.015668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:51.015668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:51.015668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:51.015668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:51.015668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:51.015668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:51.015668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:51.015668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:51.015668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:51.016767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:51.016767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:51.016767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:51.016767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:51.016767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:51.016767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:51.016767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:51.016767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:51.016767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:51.016767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:51.016767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:51.016767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:51.016767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:51.016767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:51.016767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:51.016767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:51.016767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:51.017699 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:51.017699 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:51.017699 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:51.017699 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:51.017699 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 280/576] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:51.017969 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f303a5-0440-4fdc-bb56-b7bf2243cff3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:51.048611 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd75ccb-1086-486c-8bd7-27d91a3418ae None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:51.082510 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509970b-4b31-4f85-9c70-302baecaeeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:51.083131 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509970b-4b31-4f85-9c70-302baecaeeae None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:51.083414 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509970b-4b31-4f85-9c70-302baecaeeae None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:51.083728 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509970b-4b31-4f85-9c70-302baecaeeae None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:51.084429 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509970b-4b31-4f85-9c70-302baecaeeae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:51.084735 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509970b-4b31-4f85-9c70-302baecaeeae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:51.086128 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509970b-4b31-4f85-9c70-302baecaeeae None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:51.098684 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 297/577] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:51.099509 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509970b-4b31-4f85-9c70-302baecaeeae None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:51.118983 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:51.123584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f303a5-0440-4fdc-bb56-b7bf2243cff3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:51.134950 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:51.160602 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce611f9-6e1f-4dd1-ad53-d82961627008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:51.161197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce611f9-6e1f-4dd1-ad53-d82961627008 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:51.161484 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce611f9-6e1f-4dd1-ad53-d82961627008 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:51.161762 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce611f9-6e1f-4dd1-ad53-d82961627008 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:51.172410 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce611f9-6e1f-4dd1-ad53-d82961627008 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:51.177737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-6ce611f9-6e1f-4dd1-ad53-d82961627008 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:51.177737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:51.177737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:51.177737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:51.177737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:51.177737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:51.177737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:51.177737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:51.177737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:51.177737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:51.177737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:51.177737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:51.177737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:51.177737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:51.177737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:51.177737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:51.178794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:51.178794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:51.178794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:51.178794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:51.178794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:51.178794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:51.178794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:51.178794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:51.178794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:51.178794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:51.178794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:51.178794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:51.178794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:51.178794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:51.178794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:51.178794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:51.179681 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:51.179681 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:51.179681 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:51.179681 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:51.179681 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:51.179681 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:51.179681 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:51.179681 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:51.179681 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:51.179681 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:51.179681 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:51.179681 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:51.179681 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 281/578] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jul 26 04:46:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:51.180474 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce611f9-6e1f-4dd1-ad53-d82961627008 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:51.222974 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509970b-4b31-4f85-9c70-302baecaeeae None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:51.265485 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117c44f-c73f-456a-ba6d-a5f6579af1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:51.266166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117c44f-c73f-456a-ba6d-a5f6579af1a8 None admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:51.266451 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117c44f-c73f-456a-ba6d-a5f6579af1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:51.266729 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117c44f-c73f-456a-ba6d-a5f6579af1a8 None admin] PATH_INFO: `/v3/projects/d583e0be8ef74766bffd6c3c07af18f1/users/2faeed9002ee4154bc980c146cd06bcc/roles/e5048f4e7c6b4494a15d628758090380` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:51.283089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e117c44f-c73f-456a-ba6d-a5f6579af1a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=d583e0be8ef74766bffd6c3c07af18f1, user_id=2faeed9002ee4154bc980c146cd06bcc, role_id=e5048f4e7c6b4494a15d628758090380)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:51.284631 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e117c44f-c73f-456a-ba6d-a5f6579af1a8 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:51.298660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce611f9-6e1f-4dd1-ad53-d82961627008 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:51.315612 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 298/579] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jul 26 04:46:51 2022] PUT /identity/v3/projects/d583e0be8ef74766bffd6c3c07af18f1/users/2faeed9002ee4154bc980c146cd06bcc/roles/e5048f4e7c6b4494a15d628758090380 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:46:51.315612 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117c44f-c73f-456a-ba6d-a5f6579af1a8 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:51.353490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2174aac-21c9-49a3-b304-ece5ff82c1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:51.354352 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2174aac-21c9-49a3-b304-ece5ff82c1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:51.354892 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2174aac-21c9-49a3-b304-ece5ff82c1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:51.355203 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2174aac-21c9-49a3-b304-ece5ff82c1e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:51.363395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2174aac-21c9-49a3-b304-ece5ff82c1e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:51.365141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2174aac-21c9-49a3-b304-ece5ff82c1e4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:51.378362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c2174aac-21c9-49a3-b304-ece5ff82c1e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:46:51.378362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:51.378362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:51.378362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:51.378362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:51.378362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:51.378362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:51.378362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:51.378362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:51.378362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:51.378362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:51.378362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:51.378362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:46:51.378362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:46:51.378362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:46:51.379582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:46:51.379582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:51.379582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:51.379582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:51.379582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:51.379582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:51.379582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:51.379582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:51.379582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:51.379582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:51.379582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:51.379582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:51.379582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:51.379582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:51.379582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:51.379582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:51.379582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:51.380755 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:46:51.380755 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:46:51.380755 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:46:51.380755 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:46:51.380755 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:46:51.380755 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:46:51.380755 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:46:51.380755 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:51.389961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 282/580] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:46:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:51.389961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2174aac-21c9-49a3-b304-ece5ff82c1e4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:51.410738 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:51.427179 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:51.446236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117c44f-c73f-456a-ba6d-a5f6579af1a8 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:51.492501 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a0752-a18b-4d9e-87ac-ab10d58b524a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:51.493138 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a0752-a18b-4d9e-87ac-ab10d58b524a None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:51.493419 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a0752-a18b-4d9e-87ac-ab10d58b524a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:51.493695 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a0752-a18b-4d9e-87ac-ab10d58b524a None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:51.494406 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64a0752-a18b-4d9e-87ac-ab10d58b524a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:51.494709 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64a0752-a18b-4d9e-87ac-ab10d58b524a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:51.496272 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64a0752-a18b-4d9e-87ac-ab10d58b524a None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:51.509634 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 299/581] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:51.510311 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a0752-a18b-4d9e-87ac-ab10d58b524a None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:51.521566 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2174aac-21c9-49a3-b304-ece5ff82c1e4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:51.557792 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659ac9e-65b8-4dd4-ab9d-2fed2af36ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:51.558456 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659ac9e-65b8-4dd4-ab9d-2fed2af36ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:51.558941 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659ac9e-65b8-4dd4-ab9d-2fed2af36ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:51.559313 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659ac9e-65b8-4dd4-ab9d-2fed2af36ab0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:51.568980 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659ac9e-65b8-4dd4-ab9d-2fed2af36ab0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:51.570431 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659ac9e-65b8-4dd4-ab9d-2fed2af36ab0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:51.582465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-3659ac9e-65b8-4dd4-ab9d-2fed2af36ab0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 04:46:51.582465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:51.582465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:51.582465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:51.582465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:51.582465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:51.582465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:51.582465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:51.582465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:51.582465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:51.582465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:51.582465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:51.582465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:46:51.582465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:46:51.582465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:46:51.582465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:46:51.583673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:51.583673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:51.583673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:46:51.583673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:51.583673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:46:51.583673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:51.583673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:51.583673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:51.583673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:51.583673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:51.583673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:51.583673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:51.583673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:51.583673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:51.583673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:51.583673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:51.583673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:51.583673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:51.584837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:51.584837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:51.584837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:46:51.584837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:46:51.584837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:46:51.584837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:46:51.584837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:46:51.584837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:46:51.584837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 04:46:51.584837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:51.584837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 283/582] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:51.584837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659ac9e-65b8-4dd4-ab9d-2fed2af36ab0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:51.631258 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a0752-a18b-4d9e-87ac-ab10d58b524a None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:51.669742 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:51.675569 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c404a0-f587-495f-b8a5-397a21463fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:51.676213 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c404a0-f587-495f-b8a5-397a21463fcc None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:51.676495 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c404a0-f587-495f-b8a5-397a21463fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:51.676778 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c404a0-f587-495f-b8a5-397a21463fcc None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:51.677509 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c404a0-f587-495f-b8a5-397a21463fcc None admin] RBAC: Adding query filter params (name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:51.677810 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c404a0-f587-495f-b8a5-397a21463fcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:51.679209 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c404a0-f587-495f-b8a5-397a21463fcc None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:51.692598 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 300/583] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jul 26 04:46:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:51.692598 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c404a0-f587-495f-b8a5-397a21463fcc None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:51.702117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659ac9e-65b8-4dd4-ab9d-2fed2af36ab0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:51.702335 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:51.742959 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d33e19-dd9f-469f-b343-087d07b94945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:51.743803 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d33e19-dd9f-469f-b343-087d07b94945 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:51.744183 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d33e19-dd9f-469f-b343-087d07b94945 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:51.744556 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d33e19-dd9f-469f-b343-087d07b94945 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:51.745424 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d33e19-dd9f-469f-b343-087d07b94945 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:51.745833 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d33e19-dd9f-469f-b343-087d07b94945 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:51.747523 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d33e19-dd9f-469f-b343-087d07b94945 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:51.761128 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 284/584] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:46:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:51.762177 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d33e19-dd9f-469f-b343-087d07b94945 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:51.809129 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c404a0-f587-495f-b8a5-397a21463fcc None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:51.849835 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa57d5-3401-437c-bdd2-ff22e38922f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:51.851321 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa57d5-3401-437c-bdd2-ff22e38922f1 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:51.851321 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa57d5-3401-437c-bdd2-ff22e38922f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:51.851321 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa57d5-3401-437c-bdd2-ff22e38922f1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:51.859955 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa57d5-3401-437c-bdd2-ff22e38922f1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:51.861581 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa57d5-3401-437c-bdd2-ff22e38922f1 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:51.873928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-2bfa57d5-3401-437c-bdd2-ff22e38922f1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 26 04:46:51.873928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:51.873928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:51.873928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:51.873928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:51.873928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:51.873928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:51.873928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:51.873928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:51.873928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:51.873928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:51.873928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:51.873928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:46:51.873928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:46:51.873928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:46:51.873928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:46:51.875168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:51.875168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:51.875168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:46:51.875168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:51.875168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:46:51.875168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:51.875168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:51.875168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:51.875168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:51.875168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:51.875168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:51.875168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:51.875168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:51.875168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:51.875168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:51.875168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:51.875168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:51.875168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:51.876353 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:51.876353 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:51.876353 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:46:51.876353 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:46:51.876353 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:46:51.876353 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:46:51.876353 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:46:51.876353 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:46:51.876353 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 26 04:46:51.876353 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:51.878042 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 301/585] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:51.883286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d33e19-dd9f-469f-b343-087d07b94945 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:51.883959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa57d5-3401-437c-bdd2-ff22e38922f1 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:51.924967 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199239fa-90e4-48f0-87bf-c40270a928f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:51.925516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199239fa-90e4-48f0-87bf-c40270a928f8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:51.925792 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199239fa-90e4-48f0-87bf-c40270a928f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:51.926067 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199239fa-90e4-48f0-87bf-c40270a928f8 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:51.926864 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199239fa-90e4-48f0-87bf-c40270a928f8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:51.927173 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199239fa-90e4-48f0-87bf-c40270a928f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:51.928662 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199239fa-90e4-48f0-87bf-c40270a928f8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:51.937866 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:51.953145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 285/586] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Jul 26 04:46:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:51.953145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199239fa-90e4-48f0-87bf-c40270a928f8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:51.965550 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:52.003751 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa57d5-3401-437c-bdd2-ff22e38922f1 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:52.049489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd60a1d-ab33-41d8-876b-430e8511e970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:52.050157 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd60a1d-ab33-41d8-876b-430e8511e970 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:52.050441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd60a1d-ab33-41d8-876b-430e8511e970 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:52.050719 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd60a1d-ab33-41d8-876b-430e8511e970 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:52.063249 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd60a1d-ab33-41d8-876b-430e8511e970 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:52.064195 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199239fa-90e4-48f0-87bf-c40270a928f8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:52.064762 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd60a1d-ab33-41d8-876b-430e8511e970 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:52.081244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-cdd60a1d-ab33-41d8-876b-430e8511e970 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:52.081244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:52.081244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:52.081244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:52.081244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:52.081244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:52.081244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:52.081244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:52.081244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:52.081244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:52.081244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:52.081244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:52.081244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:52.081244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:52.081244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:52.082373 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:52.082373 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:52.082373 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:52.082373 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:52.082373 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:52.082373 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:52.082373 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:52.082373 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:52.082373 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:52.082373 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:52.082373 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:52.082373 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:52.082373 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:52.082373 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:52.082373 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:52.082373 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:52.082373 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:52.088303 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:52.088303 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:52.088303 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:52.088303 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:52.088303 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 302/587] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:52.088303 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd60a1d-ab33-41d8-876b-430e8511e970 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:52.112221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b39c1-3780-4bd3-8305-865ca0a4da66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:52.113147 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b39c1-3780-4bd3-8305-865ca0a4da66 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:52.113663 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b39c1-3780-4bd3-8305-865ca0a4da66 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:52.113774 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b39c1-3780-4bd3-8305-865ca0a4da66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:52.114492 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b39c1-3780-4bd3-8305-865ca0a4da66 None admin] RBAC: Adding query filter params (role.id=e5048f4e7c6b4494a15d628758090380, scope.project.id=d583e0be8ef74766bffd6c3c07af18f1, user.id=d5218319aaa5424882714bffcbeee730) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:52.114791 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b39c1-3780-4bd3-8305-865ca0a4da66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:52.116276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b39c1-3780-4bd3-8305-865ca0a4da66 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:52.131584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 286/588] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:46:51 2022] GET /identity/v3/role_assignments?user.id=d5218319aaa5424882714bffcbeee730&scope.project.id=d583e0be8ef74766bffd6c3c07af18f1&role.id=e5048f4e7c6b4494a15d628758090380 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:52.131584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b39c1-3780-4bd3-8305-865ca0a4da66 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:52.217387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd60a1d-ab33-41d8-876b-430e8511e970 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:52.233048 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:52.241112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:52.258921 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438caba-416a-45f1-a576-a6038e3cec2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:52.259802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438caba-416a-45f1-a576-a6038e3cec2c None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:52.260188 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438caba-416a-45f1-a576-a6038e3cec2c None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:52.260558 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438caba-416a-45f1-a576-a6038e3cec2c None admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:52.261695 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4438caba-416a-45f1-a576-a6038e3cec2c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:52.262111 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4438caba-416a-45f1-a576-a6038e3cec2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:52.263819 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4438caba-416a-45f1-a576-a6038e3cec2c None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:52.290056 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 303/589] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Jul 26 04:46:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:52.290056 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438caba-416a-45f1-a576-a6038e3cec2c None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:52.290472 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b39c1-3780-4bd3-8305-865ca0a4da66 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:52.346712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7324f-bff8-4aa1-b13b-7e9be2aac631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:52.347734 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7324f-bff8-4aa1-b13b-7e9be2aac631 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:52.348206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7324f-bff8-4aa1-b13b-7e9be2aac631 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:52.348639 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7324f-bff8-4aa1-b13b-7e9be2aac631 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:52.364072 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b7324f-bff8-4aa1-b13b-7e9be2aac631 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:52.365866 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b7324f-bff8-4aa1-b13b-7e9be2aac631 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:52.382456 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-22b7324f-bff8-4aa1-b13b-7e9be2aac631 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:52.382456 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:52.382456 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:52.382456 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:52.382456 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:52.382456 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:52.382456 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:52.382456 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:52.382456 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:52.382456 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:52.382456 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:52.382456 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:52.382456 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:52.382456 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:52.382456 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:52.383617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:52.383617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:52.383617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:52.383617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:52.383617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:52.383617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:52.383617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:52.383617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:52.383617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:52.383617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:52.383617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:52.383617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:52.383617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:52.383617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:52.383617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:52.383617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:52.383617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:52.384515 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:52.384515 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:52.384515 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:52.384515 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:52.386035 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 287/590] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:52.391794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7324f-bff8-4aa1-b13b-7e9be2aac631 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:52.451973 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438caba-416a-45f1-a576-a6038e3cec2c None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:52.493127 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31f0ae-a44d-44ff-b256-1ef9319f505b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:52.496552 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31f0ae-a44d-44ff-b256-1ef9319f505b None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:52.497018 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31f0ae-a44d-44ff-b256-1ef9319f505b None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:52.497313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31f0ae-a44d-44ff-b256-1ef9319f505b None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:52.498051 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f31f0ae-a44d-44ff-b256-1ef9319f505b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:52.498365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f31f0ae-a44d-44ff-b256-1ef9319f505b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:52.500513 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f31f0ae-a44d-44ff-b256-1ef9319f505b None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:52.516605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 304/591] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:52.517801 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31f0ae-a44d-44ff-b256-1ef9319f505b None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:52.533306 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7324f-bff8-4aa1-b13b-7e9be2aac631 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:52.554118 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:52.579877 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:52.587947 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c78352-d5be-4429-b8b9-93eeaf8919f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:52.589102 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c78352-d5be-4429-b8b9-93eeaf8919f6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:52.589385 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c78352-d5be-4429-b8b9-93eeaf8919f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:52.590099 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c78352-d5be-4429-b8b9-93eeaf8919f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:52.606865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c78352-d5be-4429-b8b9-93eeaf8919f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:52.608559 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c78352-d5be-4429-b8b9-93eeaf8919f6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:52.629370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-94c78352-d5be-4429-b8b9-93eeaf8919f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:52.629370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:52.629370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:52.629370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:52.629370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:52.629370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:52.629370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:52.629370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:52.629370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:52.629370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:52.629370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:52.629370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:52.629370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:52.629370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:52.629370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:52.630321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:52.630321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:52.630321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:52.630321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:52.630321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:52.630321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:52.630321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:52.630321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:52.630321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:52.630321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:52.630321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:52.630321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:52.630321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:52.630321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:52.630321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:52.630321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:52.630321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:52.631226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:52.631226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:52.631226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:52.631226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:52.633686 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 288/592] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:52.633686 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c78352-d5be-4429-b8b9-93eeaf8919f6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:52.655892 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31f0ae-a44d-44ff-b256-1ef9319f505b None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:52.695036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569fddd-dcaf-46fb-aee1-265f170a4420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:52.695915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569fddd-dcaf-46fb-aee1-265f170a4420 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:52.696207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569fddd-dcaf-46fb-aee1-265f170a4420 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:52.696483 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569fddd-dcaf-46fb-aee1-265f170a4420 None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:52.697799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e569fddd-dcaf-46fb-aee1-265f170a4420 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:52.698272 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e569fddd-dcaf-46fb-aee1-265f170a4420 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:52.700071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e569fddd-dcaf-46fb-aee1-265f170a4420 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:52.716356 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 305/593] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:52.718093 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569fddd-dcaf-46fb-aee1-265f170a4420 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:52.789877 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c78352-d5be-4429-b8b9-93eeaf8919f6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:52.832615 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:52.835305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1736a-f8d1-4186-94f5-430203813dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:52.836165 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1736a-f8d1-4186-94f5-430203813dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:52.836461 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1736a-f8d1-4186-94f5-430203813dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:52.836742 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1736a-f8d1-4186-94f5-430203813dd6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:52.839605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569fddd-dcaf-46fb-aee1-265f170a4420 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:52.846205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1736a-f8d1-4186-94f5-430203813dd6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:52.847743 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1736a-f8d1-4186-94f5-430203813dd6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:52.860540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-22b1736a-f8d1-4186-94f5-430203813dd6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 04:46:52.860540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:52.860540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:52.860540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:52.860540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:52.860540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:52.860540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:52.860540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:52.860540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:52.860540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:52.860540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:52.860540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:52.860540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:46:52.860540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:46:52.860540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:46:52.860540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:46:52.861754 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:52.861754 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:52.861754 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:46:52.861754 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:52.861754 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:46:52.861754 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:52.861754 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:52.861754 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:52.861754 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:52.861754 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:52.861754 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:52.861754 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:52.861754 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:52.861754 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:52.861754 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:52.861754 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:52.861754 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:52.861754 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:52.862835 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:52.862835 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:52.862835 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:46:52.862835 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:46:52.862835 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:46:52.862835 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:46:52.862835 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:46:52.862835 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:46:52.862835 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 04:46:52.862835 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:52.862835 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 289/594] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jul 26 04:46:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:52.863411 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1736a-f8d1-4186-94f5-430203813dd6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:52.883708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c393f-f5c4-4962-b0a5-4fb3f25c436c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:52.884699 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c393f-f5c4-4962-b0a5-4fb3f25c436c None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:52.885140 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c393f-f5c4-4962-b0a5-4fb3f25c436c None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:52.885558 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c393f-f5c4-4962-b0a5-4fb3f25c436c None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:52.886415 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c393f-f5c4-4962-b0a5-4fb3f25c436c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:52.886907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c393f-f5c4-4962-b0a5-4fb3f25c436c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:52.888618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c393f-f5c4-4962-b0a5-4fb3f25c436c None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:52.890525 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:52.904772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 306/595] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:52.905828 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c393f-f5c4-4962-b0a5-4fb3f25c436c None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:52.973907 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1736a-f8d1-4186-94f5-430203813dd6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:53.010072 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364ff26-427c-4338-a855-05182b7a20c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:53.011009 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364ff26-427c-4338-a855-05182b7a20c9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:53.011438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364ff26-427c-4338-a855-05182b7a20c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:53.011912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364ff26-427c-4338-a855-05182b7a20c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:53.020555 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364ff26-427c-4338-a855-05182b7a20c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:53.022374 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364ff26-427c-4338-a855-05182b7a20c9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:53.029179 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c393f-f5c4-4962-b0a5-4fb3f25c436c None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:53.035839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-6364ff26-427c-4338-a855-05182b7a20c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:46:53.035839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:53.035839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:53.035839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:53.035839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:53.035839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:53.035839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:53.035839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:53.035839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:53.035839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:53.035839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:53.035839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:53.035839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:46:53.035839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:46:53.035839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:46:53.036769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:46:53.036769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:53.036769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:53.036769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:53.036769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:53.036769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:53.036769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:53.036769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:53.036769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:53.036769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:53.036769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:53.036769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:53.036769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:53.036769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:53.036769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:53.036769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:53.036769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:53.037669 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:46:53.037669 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:46:53.037669 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:46:53.037669 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:46:53.037669 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:46:53.037669 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:46:53.037669 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:46:53.037669 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:53.040354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 290/596] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:46:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:53.041387 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364ff26-427c-4338-a855-05182b7a20c9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:53.071253 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483521b-3fa1-4878-861e-0a5b4329c3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:53.072330 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483521b-3fa1-4878-861e-0a5b4329c3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:53.072467 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483521b-3fa1-4878-861e-0a5b4329c3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:53.072718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483521b-3fa1-4878-861e-0a5b4329c3f2 None admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:53.073561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7483521b-3fa1-4878-861e-0a5b4329c3f2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:53.073861 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7483521b-3fa1-4878-861e-0a5b4329c3f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:53.075487 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7483521b-3fa1-4878-861e-0a5b4329c3f2 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:53.097519 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 307/597] 10.222.0.17 () {56 vars in 1139 bytes} [Tue Jul 26 04:46:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:53.098407 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483521b-3fa1-4878-861e-0a5b4329c3f2 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:53.123155 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:53.134464 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:53.151487 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364ff26-427c-4338-a855-05182b7a20c9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:53.188282 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053d39a-7371-4355-a1e2-2a09d86b253f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:53.189239 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053d39a-7371-4355-a1e2-2a09d86b253f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:53.189694 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053d39a-7371-4355-a1e2-2a09d86b253f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:53.190158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053d39a-7371-4355-a1e2-2a09d86b253f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:53.198139 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053d39a-7371-4355-a1e2-2a09d86b253f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:53.200018 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053d39a-7371-4355-a1e2-2a09d86b253f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:53.213129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-8053d39a-7371-4355-a1e2-2a09d86b253f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:46:53.213129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:53.213129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:53.213129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:53.213129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:53.213129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:53.213129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:53.213129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:53.213129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:53.213129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:53.213129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:53.213129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:53.213129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:46:53.213129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:46:53.213129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:46:53.214244 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:46:53.214244 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:53.214244 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:53.214244 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:53.214244 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:53.214244 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:53.214244 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:53.214244 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:53.214244 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:53.214244 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:53.214244 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:53.214244 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:53.214244 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:53.214244 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:53.214244 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:53.214244 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:53.214244 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:53.215316 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:46:53.215316 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:46:53.215316 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:46:53.215316 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:46:53.215316 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:46:53.215316 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:46:53.215316 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:46:53.215316 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:53.215787 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483521b-3fa1-4878-861e-0a5b4329c3f2 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:53.218039 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 291/598] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:46:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:53.219058 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053d39a-7371-4355-a1e2-2a09d86b253f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:53.251631 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f6495-d6d8-4b9f-b0c0-416c721c0aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:53.252409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f6495-d6d8-4b9f-b0c0-416c721c0aeb None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:53.252706 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f6495-d6d8-4b9f-b0c0-416c721c0aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:53.253236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f6495-d6d8-4b9f-b0c0-416c721c0aeb None admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:53.254071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f6495-d6d8-4b9f-b0c0-416c721c0aeb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:53.254389 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f6495-d6d8-4b9f-b0c0-416c721c0aeb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:53.256138 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f6495-d6d8-4b9f-b0c0-416c721c0aeb None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:53.270001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 308/599] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:46:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:53.270985 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f6495-d6d8-4b9f-b0c0-416c721c0aeb None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:53.354170 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053d39a-7371-4355-a1e2-2a09d86b253f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:53.372554 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:53.407352 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd87e46-6875-4486-b233-d6d4ca91fdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:53.408091 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd87e46-6875-4486-b233-d6d4ca91fdb9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:53.408379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd87e46-6875-4486-b233-d6d4ca91fdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:53.408644 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd87e46-6875-4486-b233-d6d4ca91fdb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:53.415161 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f6495-d6d8-4b9f-b0c0-416c721c0aeb None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:53.423532 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd87e46-6875-4486-b233-d6d4ca91fdb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:53.425105 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd87e46-6875-4486-b233-d6d4ca91fdb9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:53.428616 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:53.445209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-acd87e46-6875-4486-b233-d6d4ca91fdb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:53.445209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:53.445209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:53.445209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:53.445209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:53.445209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:53.445209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:53.445209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:53.445209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:53.445209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:53.445209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:53.445209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:53.445209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:53.445209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:53.445209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:53.446351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:53.446351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:53.446351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:53.446351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:53.446351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:53.446351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:53.446351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:53.446351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:53.446351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:53.446351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:53.446351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:53.446351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:53.446351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:53.446351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:53.446351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:53.446351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:53.446351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:53.447350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:53.447350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:53.447350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:53.447350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:53.447350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 292/600] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:53.447677 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd87e46-6875-4486-b233-d6d4ca91fdb9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:53.465468 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b49772-b54a-44b2-8c3a-1a4260ee979e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:53.466288 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b49772-b54a-44b2-8c3a-1a4260ee979e None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:53.466612 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b49772-b54a-44b2-8c3a-1a4260ee979e None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:53.467317 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b49772-b54a-44b2-8c3a-1a4260ee979e None admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:53.468316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b49772-b54a-44b2-8c3a-1a4260ee979e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:53.468690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b49772-b54a-44b2-8c3a-1a4260ee979e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:53.470222 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b49772-b54a-44b2-8c3a-1a4260ee979e None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:53.487439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 309/601] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:46:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:53.487439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51cd215-4e89-45b1-86df-19d05b94b06f None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:53.487439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51cd215-4e89-45b1-86df-19d05b94b06f None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:53.487439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51cd215-4e89-45b1-86df-19d05b94b06f None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:53.487439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 310/602] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:53.487439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51cd215-4e89-45b1-86df-19d05b94b06f None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:53.571190 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd87e46-6875-4486-b233-d6d4ca91fdb9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:53.600140 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51cd215-4e89-45b1-86df-19d05b94b06f None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:53.607030 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d95a3d4-da3a-4a3a-85ac-c0d7450a7713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:53.607664 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95a3d4-da3a-4a3a-85ac-c0d7450a7713 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:53.607949 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95a3d4-da3a-4a3a-85ac-c0d7450a7713 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:53.608224 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95a3d4-da3a-4a3a-85ac-c0d7450a7713 None admin] PATH_INFO: `/v3/projects/d583e0be8ef74766bffd6c3c07af18f1/users/79ef3d83ac454fac9f890374cd22507a/roles/e5048f4e7c6b4494a15d628758090380` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:53.623864 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d95a3d4-da3a-4a3a-85ac-c0d7450a7713 None admin] RBAC: Authorizing `identity:create_grant(project_id=d583e0be8ef74766bffd6c3c07af18f1, user_id=79ef3d83ac454fac9f890374cd22507a, role_id=e5048f4e7c6b4494a15d628758090380)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:53.625318 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d95a3d4-da3a-4a3a-85ac-c0d7450a7713 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:53.637102 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6a1f5-9e20-4d08-825c-15a22c187271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:53.637746 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6a1f5-9e20-4d08-825c-15a22c187271 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:53.638024 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6a1f5-9e20-4d08-825c-15a22c187271 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:53.638301 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6a1f5-9e20-4d08-825c-15a22c187271 None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:53.647895 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6a1f5-9e20-4d08-825c-15a22c187271 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:53.648099 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 293/603] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jul 26 04:46:53 2022] PUT /identity/v3/projects/d583e0be8ef74766bffd6c3c07af18f1/users/79ef3d83ac454fac9f890374cd22507a/roles/e5048f4e7c6b4494a15d628758090380 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:46:53.648207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6a1f5-9e20-4d08-825c-15a22c187271 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:53.648281 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e541dfa-61e1-49a9-bbdb-03172c84503c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:53.648281 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e541dfa-61e1-49a9-bbdb-03172c84503c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:53.648281 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e541dfa-61e1-49a9-bbdb-03172c84503c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:53.648454 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6a1f5-9e20-4d08-825c-15a22c187271 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:53.650568 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:53.662890 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 311/604] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:53.663945 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6a1f5-9e20-4d08-825c-15a22c187271 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:53.672119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1e541dfa-61e1-49a9-bbdb-03172c84503c None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:53.673370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1e541dfa-61e1-49a9-bbdb-03172c84503c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:53.745827 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:53.794618 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 294/605] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:53.795591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e541dfa-61e1-49a9-bbdb-03172c84503c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:53.805897 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6a1f5-9e20-4d08-825c-15a22c187271 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:53.851652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01365268-047a-4ad1-8c1d-f72bcd54e4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:53.852360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01365268-047a-4ad1-8c1d-f72bcd54e4bf None admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:53.852640 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01365268-047a-4ad1-8c1d-f72bcd54e4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:53.852917 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01365268-047a-4ad1-8c1d-f72bcd54e4bf None admin] PATH_INFO: `/v3/projects/d583e0be8ef74766bffd6c3c07af18f1/users/6946a788d2ef49ff873ca8e81990527e/roles/e5048f4e7c6b4494a15d628758090380` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:53.870146 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01365268-047a-4ad1-8c1d-f72bcd54e4bf None admin] RBAC: Authorizing `identity:create_grant(project_id=d583e0be8ef74766bffd6c3c07af18f1, user_id=6946a788d2ef49ff873ca8e81990527e, role_id=e5048f4e7c6b4494a15d628758090380)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:53.871762 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01365268-047a-4ad1-8c1d-f72bcd54e4bf None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:53.895668 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 312/606] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jul 26 04:46:53 2022] PUT /identity/v3/projects/d583e0be8ef74766bffd6c3c07af18f1/users/6946a788d2ef49ff873ca8e81990527e/roles/e5048f4e7c6b4494a15d628758090380 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:46:53.896745 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01365268-047a-4ad1-8c1d-f72bcd54e4bf None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:53.946516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e541dfa-61e1-49a9-bbdb-03172c84503c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:53.969556 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:54.012811 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d7327e-cc1b-4cbe-a049-8e65e6f0506a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:54.013690 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7327e-cc1b-4cbe-a049-8e65e6f0506a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:54.014022 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7327e-cc1b-4cbe-a049-8e65e6f0506a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:54.014309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7327e-cc1b-4cbe-a049-8e65e6f0506a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:54.028676 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d7327e-cc1b-4cbe-a049-8e65e6f0506a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:54.030320 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d7327e-cc1b-4cbe-a049-8e65e6f0506a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:54.053153 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01365268-047a-4ad1-8c1d-f72bcd54e4bf None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:54.059350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b3d7327e-cc1b-4cbe-a049-8e65e6f0506a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:46:54.059350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:54.059350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:54.059350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:54.059350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:54.059350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:54.059350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:54.059350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:54.059350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:54.059350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:54.059350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:54.059350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:54.059350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:46:54.059350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:46:54.059350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:46:54.060529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:46:54.060529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:54.060529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:54.060529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:54.060529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:54.060529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:54.060529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:54.060529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:54.060529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:54.060529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:54.060529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:54.060529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:54.060529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:54.060529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:54.060529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:54.060529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:54.060529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:54.061513 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:46:54.061513 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:46:54.061513 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:46:54.061513 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:46:54.061513 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:46:54.061513 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:46:54.061513 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:46:54.061513 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:54.067525 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 295/607] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:46:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:54.074451 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d7327e-cc1b-4cbe-a049-8e65e6f0506a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:54.115938 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3cd80a-54f1-40b5-a7a0-d275730772d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:54.116990 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3cd80a-54f1-40b5-a7a0-d275730772d4 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:54.117427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3cd80a-54f1-40b5-a7a0-d275730772d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:54.117862 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3cd80a-54f1-40b5-a7a0-d275730772d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:54.153758 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3cd80a-54f1-40b5-a7a0-d275730772d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:54.162673 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-8e3cd80a-54f1-40b5-a7a0-d275730772d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:54.162673 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:54.162673 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:54.162673 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:54.162673 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:54.162673 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:54.162673 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:54.162673 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:54.162673 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:54.162673 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:54.162673 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:54.162673 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:54.162673 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:54.162673 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:54.162673 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:54.162673 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:54.163956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:54.163956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:54.163956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:54.163956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:54.163956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:54.163956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:54.163956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:54.163956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:54.163956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:54.163956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:54.163956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:54.163956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:54.163956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:54.163956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:54.163956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:54.163956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:54.164930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:54.164930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:54.164930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:54.164930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:54.164930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:54.164930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:54.164930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:54.164930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:54.164930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:54.164930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:54.164930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:54.164930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:54.168087 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 313/608] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jul 26 04:46:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:54.172534 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:54.173983 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3cd80a-54f1-40b5-a7a0-d275730772d4 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:54.298126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d7327e-cc1b-4cbe-a049-8e65e6f0506a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:54.308094 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3cd80a-54f1-40b5-a7a0-d275730772d4 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:54.324019 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:54.343162 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941fb528-a52d-466c-bdee-be33ada29580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:54.343944 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fb528-a52d-466c-bdee-be33ada29580 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:54.344238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fb528-a52d-466c-bdee-be33ada29580 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:54.344519 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fb528-a52d-466c-bdee-be33ada29580 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:54.345295 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941fb528-a52d-466c-bdee-be33ada29580 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:54.345604 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941fb528-a52d-466c-bdee-be33ada29580 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:54.347310 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941fb528-a52d-466c-bdee-be33ada29580 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:54.349767 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8acff0-c776-48b9-b522-00150087a960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:54.350696 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8acff0-c776-48b9-b522-00150087a960 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:54.351131 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8acff0-c776-48b9-b522-00150087a960 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:54.351581 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8acff0-c776-48b9-b522-00150087a960 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:54.352470 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8acff0-c776-48b9-b522-00150087a960 None admin] RBAC: Adding query filter params (name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:54.352924 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8acff0-c776-48b9-b522-00150087a960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:54.354485 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8acff0-c776-48b9-b522-00150087a960 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:54.361075 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 296/609] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:46:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:54.366873 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce57dd2-a839-47c1-865d-562f2fd1dd6c None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:54.366873 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce57dd2-a839-47c1-865d-562f2fd1dd6c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:54.366873 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce57dd2-a839-47c1-865d-562f2fd1dd6c None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:54.366873 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 297/610] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:54.367930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce57dd2-a839-47c1-865d-562f2fd1dd6c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:54.370333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 314/611] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jul 26 04:46:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:54.372304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045f526-2801-4cff-9220-77e012f16e36 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:54.372839 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045f526-2801-4cff-9220-77e012f16e36 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:54.373295 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045f526-2801-4cff-9220-77e012f16e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:54.393716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-7045f526-2801-4cff-9220-77e012f16e36 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:54.395161 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-7045f526-2801-4cff-9220-77e012f16e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:54.485634 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce57dd2-a839-47c1-865d-562f2fd1dd6c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:54.491837 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 315/612] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:54.492851 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045f526-2801-4cff-9220-77e012f16e36 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:54.504150 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:54.523471 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076fdef-b598-4464-bc19-14ab07d6285f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:54.524314 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076fdef-b598-4464-bc19-14ab07d6285f None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:54.524608 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076fdef-b598-4464-bc19-14ab07d6285f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:54.524888 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076fdef-b598-4464-bc19-14ab07d6285f None admin] PATH_INFO: `/v3/projects/d583e0be8ef74766bffd6c3c07af18f1/users/dbdb8997891241e9b2e829feb8023f8d/roles/e5048f4e7c6b4494a15d628758090380` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:54.541885 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076fdef-b598-4464-bc19-14ab07d6285f None admin] RBAC: Authorizing `identity:create_grant(project_id=d583e0be8ef74766bffd6c3c07af18f1, user_id=dbdb8997891241e9b2e829feb8023f8d, role_id=e5048f4e7c6b4494a15d628758090380)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:54.543991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076fdef-b598-4464-bc19-14ab07d6285f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:54.565848 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 298/613] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jul 26 04:46:54 2022] PUT /identity/v3/projects/d583e0be8ef74766bffd6c3c07af18f1/users/dbdb8997891241e9b2e829feb8023f8d/roles/e5048f4e7c6b4494a15d628758090380 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:46:54.566971 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076fdef-b598-4464-bc19-14ab07d6285f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:54.616404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:54.622096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045f526-2801-4cff-9220-77e012f16e36 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:54.660642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2de69-6af4-43d3-bdf7-7c1ebefd6edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:54.661688 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2de69-6af4-43d3-bdf7-7c1ebefd6edc None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:54.662265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2de69-6af4-43d3-bdf7-7c1ebefd6edc None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:54.662851 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2de69-6af4-43d3-bdf7-7c1ebefd6edc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:54.675067 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2de69-6af4-43d3-bdf7-7c1ebefd6edc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:54.677290 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2de69-6af4-43d3-bdf7-7c1ebefd6edc None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:54.686815 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076fdef-b598-4464-bc19-14ab07d6285f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:54.695358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-67b2de69-6af4-43d3-bdf7-7c1ebefd6edc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:54.695358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:54.695358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:54.695358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:54.695358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:54.695358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:54.695358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:54.695358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:54.695358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:54.695358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:54.695358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:54.695358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:54.695358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:54.695358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:54.695358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:54.696327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:54.696327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:54.696327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:54.696327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:54.696327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:54.696327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:54.696327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:54.696327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:54.696327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:54.696327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:54.696327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:54.696327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:54.696327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:54.696327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:54.696327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:54.696327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:54.696327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:54.697238 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:54.697238 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:54.697238 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:54.697238 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:54.699714 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 316/614] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:54.705445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2de69-6af4-43d3-bdf7-7c1ebefd6edc None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:54.758337 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616edc8-b354-4f0e-b45b-196173fd5364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:54.758920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616edc8-b354-4f0e-b45b-196173fd5364 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:54.759206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616edc8-b354-4f0e-b45b-196173fd5364 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:54.759485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616edc8-b354-4f0e-b45b-196173fd5364 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:54.785423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b616edc8-b354-4f0e-b45b-196173fd5364 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:54.790562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b616edc8-b354-4f0e-b45b-196173fd5364 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:54.790562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:54.790562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:54.790562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:54.790562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:54.790562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:54.790562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:54.790562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:54.790562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:54.790562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:54.790562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:54.790562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:54.790562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:54.790562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:54.790562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:54.790562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:54.791535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:54.791535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:54.791535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:54.791535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:54.791535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:54.791535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:54.791535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:54.791535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:54.791535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:54.791535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:54.791535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:54.791535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:54.791535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:54.791535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:54.791535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:54.791535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:54.792405 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:54.792405 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:54.792405 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:54.792405 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:54.792405 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:54.792405 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:54.792405 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:54.792405 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:54.792405 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:54.792405 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:54.792405 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:54.792405 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:54.797831 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 299/615] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jul 26 04:46:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:54.811341 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616edc8-b354-4f0e-b45b-196173fd5364 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:54.823324 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:54.833181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2de69-6af4-43d3-bdf7-7c1ebefd6edc None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:54.871723 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e032d2-a0da-445d-8b40-275c1025f634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:54.872377 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e032d2-a0da-445d-8b40-275c1025f634 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:54.872658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e032d2-a0da-445d-8b40-275c1025f634 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:54.873000 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e032d2-a0da-445d-8b40-275c1025f634 None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:54.873842 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e032d2-a0da-445d-8b40-275c1025f634 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:54.874151 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e032d2-a0da-445d-8b40-275c1025f634 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:54.875684 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e032d2-a0da-445d-8b40-275c1025f634 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:54.889800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 317/616] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:54.895664 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e032d2-a0da-445d-8b40-275c1025f634 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:54.914478 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:54.937720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616edc8-b354-4f0e-b45b-196173fd5364 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:54.986837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc10f2f-948d-4cd1-ace0-3d7402801b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:54.987861 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc10f2f-948d-4cd1-ace0-3d7402801b98 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:54.988445 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc10f2f-948d-4cd1-ace0-3d7402801b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:54.989010 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc10f2f-948d-4cd1-ace0-3d7402801b98 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:54.990369 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc10f2f-948d-4cd1-ace0-3d7402801b98 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:54.990867 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc10f2f-948d-4cd1-ace0-3d7402801b98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:54.993549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc10f2f-948d-4cd1-ace0-3d7402801b98 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:55.009535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 300/617] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jul 26 04:46:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:55.016102 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc10f2f-948d-4cd1-ace0-3d7402801b98 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:55.046104 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e032d2-a0da-445d-8b40-275c1025f634 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:55.133891 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:55.143245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5edaadc-851f-406c-b01d-948329f1c6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:55.143944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edaadc-851f-406c-b01d-948329f1c6d2 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:55.144231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edaadc-851f-406c-b01d-948329f1c6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:55.144510 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edaadc-851f-406c-b01d-948329f1c6d2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:55.168503 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5edaadc-851f-406c-b01d-948329f1c6d2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:55.170029 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5edaadc-851f-406c-b01d-948329f1c6d2 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:55.173519 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc10f2f-948d-4cd1-ace0-3d7402801b98 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:55.196891 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-d5edaadc-851f-406c-b01d-948329f1c6d2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 04:46:55.196891 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:55.196891 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:55.196891 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:55.196891 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:55.196891 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:55.196891 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:55.196891 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:55.196891 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:55.196891 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:55.196891 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:55.196891 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:55.196891 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:46:55.196891 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:46:55.196891 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:46:55.196891 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:46:55.197909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:55.197909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:55.197909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:46:55.197909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:55.197909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:46:55.197909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:55.197909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:55.197909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:55.197909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:55.197909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:55.197909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:55.197909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:55.197909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:55.197909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:55.197909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:55.197909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:55.197909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:55.197909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:55.199000 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:55.199000 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:55.199000 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:46:55.199000 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:46:55.199000 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:46:55.199000 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:46:55.199000 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:46:55.199000 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:46:55.199000 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 04:46:55.199000 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:55.200063 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 318/618] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Jul 26 04:46:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:55.208630 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5edaadc-851f-406c-b01d-948329f1c6d2 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:55.229964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a7d99-f596-4701-8f45-4954c4db1dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:55.230679 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a7d99-f596-4701-8f45-4954c4db1dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:55.230964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a7d99-f596-4701-8f45-4954c4db1dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:55.231240 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a7d99-f596-4701-8f45-4954c4db1dd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:55.248794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a7d99-f596-4701-8f45-4954c4db1dd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:55.250281 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a7d99-f596-4701-8f45-4954c4db1dd3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:55.274122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b32a7d99-f596-4701-8f45-4954c4db1dd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:55.274122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:55.274122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:55.274122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:55.274122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:55.274122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:55.274122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:55.274122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:55.274122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:55.274122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:55.274122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:55.274122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:55.274122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:55.274122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:55.274122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:55.275077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:55.275077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:55.275077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:55.275077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:55.275077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:55.275077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:55.275077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:55.275077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:55.275077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:55.275077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:55.275077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:55.275077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:55.275077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:55.275077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:55.275077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:55.275077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:55.275077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:55.276174 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:55.276174 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:55.276174 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:55.276174 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:55.276174 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 301/619] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:55.283854 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a7d99-f596-4701-8f45-4954c4db1dd3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:55.368839 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:55.463096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5edaadc-851f-406c-b01d-948329f1c6d2 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:55.536583 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a7d99-f596-4701-8f45-4954c4db1dd3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:55.543442 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f436f5d-011f-4132-b615-f98500315011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:55.544132 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f436f5d-011f-4132-b615-f98500315011 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:55.544413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f436f5d-011f-4132-b615-f98500315011 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:55.544692 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f436f5d-011f-4132-b615-f98500315011 None admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:55.545500 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f436f5d-011f-4132-b615-f98500315011 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:55.545805 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f436f5d-011f-4132-b615-f98500315011 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:55.547261 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f436f5d-011f-4132-b615-f98500315011 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:55.579693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:55.587113 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 319/620] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Jul 26 04:46:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:55.593243 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f436f5d-011f-4132-b615-f98500315011 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:55.616409 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5d994-547f-49d5-b85a-d1d246e518f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:55.617062 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5d994-547f-49d5-b85a-d1d246e518f3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:55.617342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5d994-547f-49d5-b85a-d1d246e518f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:55.617623 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5d994-547f-49d5-b85a-d1d246e518f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:55.618340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5d994-547f-49d5-b85a-d1d246e518f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:55.618640 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5d994-547f-49d5-b85a-d1d246e518f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:55.620082 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5d994-547f-49d5-b85a-d1d246e518f3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:55.648014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 302/621] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:55.654671 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5d994-547f-49d5-b85a-d1d246e518f3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:55.795839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5d994-547f-49d5-b85a-d1d246e518f3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:55.811598 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f436f5d-011f-4132-b615-f98500315011 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:55.844573 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92d4d7-65c2-48e5-8fc1-c42cb969661b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:55.845268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92d4d7-65c2-48e5-8fc1-c42cb969661b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:55.845549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92d4d7-65c2-48e5-8fc1-c42cb969661b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:55.845906 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92d4d7-65c2-48e5-8fc1-c42cb969661b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:55.854920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c92d4d7-65c2-48e5-8fc1-c42cb969661b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:55.856600 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c92d4d7-65c2-48e5-8fc1-c42cb969661b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:55.858313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:55.877584 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d86c5-6270-4ffd-93d0-56a90d26c8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:55.878253 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d86c5-6270-4ffd-93d0-56a90d26c8ca None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:55.878531 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d86c5-6270-4ffd-93d0-56a90d26c8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:55.878798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d86c5-6270-4ffd-93d0-56a90d26c8ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:55.879765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-0c92d4d7-65c2-48e5-8fc1-c42cb969661b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 04:46:55.879765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:55.879765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:55.879765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:55.879765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:55.879765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:55.879765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:55.879765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:55.879765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:55.879765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:55.879765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:55.879765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:55.879765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:46:55.879765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:46:55.879765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:46:55.879765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:46:55.880668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:55.880668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:55.880668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:46:55.880668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:55.880668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:46:55.880668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:55.880668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:55.880668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:55.880668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:55.880668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:55.880668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:55.880668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:55.880668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:55.880668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:55.880668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:55.880668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:55.880668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:55.880668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:55.881605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:55.881605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:55.881605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:46:55.881605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:46:55.881605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:46:55.881605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:46:55.881605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:46:55.881605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:46:55.881605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 04:46:55.881605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:55.888465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 303/622] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:46:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:55.890028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92d4d7-65c2-48e5-8fc1-c42cb969661b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:55.976895 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:56.040749 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d86c5-6270-4ffd-93d0-56a90d26c8ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:56.042369 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d86c5-6270-4ffd-93d0-56a90d26c8ca None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:56.043256 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92d4d7-65c2-48e5-8fc1-c42cb969661b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:56.079502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dec894-662f-4733-ac9a-0c07def38be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:56.080315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dec894-662f-4733-ac9a-0c07def38be6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:56.080621 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dec894-662f-4733-ac9a-0c07def38be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:56.080960 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dec894-662f-4733-ac9a-0c07def38be6 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:56.081922 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dec894-662f-4733-ac9a-0c07def38be6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:56.082227 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dec894-662f-4733-ac9a-0c07def38be6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:56.083723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dec894-662f-4733-ac9a-0c07def38be6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:56.104507 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 304/623] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Jul 26 04:46:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:46:56.105962 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b3f7b-98ae-49e1-a789-e05e308bf181 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:56.106240 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b3f7b-98ae-49e1-a789-e05e308bf181 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:56.106519 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b3f7b-98ae-49e1-a789-e05e308bf181 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:56.116422 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 305/624] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:56.116422 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b3f7b-98ae-49e1-a789-e05e308bf181 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:56.197585 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 320/625] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 04:46:56.199017 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439cf73c-1597-487a-bddb-1f80729639ac None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:56.199314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439cf73c-1597-487a-bddb-1f80729639ac None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:56.199618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439cf73c-1597-487a-bddb-1f80729639ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:56.213113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:56.219618 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b3f7b-98ae-49e1-a789-e05e308bf181 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:56.221305 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-439cf73c-1597-487a-bddb-1f80729639ac None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:56.222782 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-439cf73c-1597-487a-bddb-1f80729639ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:56.256442 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:56.256690 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd94aed-f645-460d-8c54-8919f5267237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:56.257364 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd94aed-f645-460d-8c54-8919f5267237 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:56.257630 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd94aed-f645-460d-8c54-8919f5267237 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:56.257903 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd94aed-f645-460d-8c54-8919f5267237 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:56.271211 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd94aed-f645-460d-8c54-8919f5267237 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:56.273046 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd94aed-f645-460d-8c54-8919f5267237 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:56.292096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-2bd94aed-f645-460d-8c54-8919f5267237 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:56.292096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:56.292096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:56.292096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:56.292096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:56.292096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:56.292096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:56.292096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:56.292096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:56.292096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:56.292096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:56.292096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:56.292096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:56.292096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:56.292096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:56.293111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:56.293111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:56.293111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:56.293111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:56.293111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:56.293111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:56.293111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:56.293111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:56.293111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:56.293111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:56.293111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:56.293111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:56.293111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:56.293111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:56.293111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:56.293111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:56.293111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:56.294034 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:56.294034 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:56.294034 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:56.294034 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:56.296621 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 306/626] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:56.297967 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd94aed-f645-460d-8c54-8919f5267237 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:56.306231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 321/627] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:56.307657 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439cf73c-1597-487a-bddb-1f80729639ac None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:56.418109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd94aed-f645-460d-8c54-8919f5267237 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:56.424785 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439cf73c-1597-487a-bddb-1f80729639ac None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:56.453505 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d893915-8d2e-470b-b529-60f386509cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:56.454166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d893915-8d2e-470b-b529-60f386509cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:56.454450 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d893915-8d2e-470b-b529-60f386509cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:56.454724 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d893915-8d2e-470b-b529-60f386509cc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:56.461041 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebcc64-9df9-4913-b40e-34aa92c5a69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:56.461669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebcc64-9df9-4913-b40e-34aa92c5a69a None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:56.462376 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebcc64-9df9-4913-b40e-34aa92c5a69a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:56.462656 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebcc64-9df9-4913-b40e-34aa92c5a69a None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:56.463738 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebcc64-9df9-4913-b40e-34aa92c5a69a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:56.464073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebcc64-9df9-4913-b40e-34aa92c5a69a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:56.465531 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebcc64-9df9-4913-b40e-34aa92c5a69a None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:56.465727 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d893915-8d2e-470b-b529-60f386509cc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:56.467370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d893915-8d2e-470b-b529-60f386509cc3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:56.482315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 322/628] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:56.482315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebcc64-9df9-4913-b40e-34aa92c5a69a None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:56.482587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-1d893915-8d2e-470b-b529-60f386509cc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:56.482587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:56.482587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:56.482587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:56.482587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:56.482587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:56.482587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:56.482587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:56.482587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:56.482587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:56.482587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:56.482587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:56.482587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:56.482587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:56.482587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:56.483379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:56.483379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:56.483379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:56.483379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:56.483379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:56.483379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:56.483379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:56.483379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:56.483379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:56.483379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:56.483379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:56.483379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:56.483379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:56.483379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:56.483379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:56.483379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:56.483379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:56.484292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:56.484292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:56.484292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:56.484292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:56.484292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 307/629] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:56.485231 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08077a7-2734-4918-a5d8-3d15926f4e13 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:56.485231 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08077a7-2734-4918-a5d8-3d15926f4e13 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:56.485231 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08077a7-2734-4918-a5d8-3d15926f4e13 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:56.486546 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 308/630] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:56.487104 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08077a7-2734-4918-a5d8-3d15926f4e13 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:56.490231 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:56.513108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:56.599129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08077a7-2734-4918-a5d8-3d15926f4e13 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:56.606070 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebcc64-9df9-4913-b40e-34aa92c5a69a None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:56.635102 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fbb6a1-c38c-4208-a63e-69fb6240ebfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:56.635785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fbb6a1-c38c-4208-a63e-69fb6240ebfa None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:56.636067 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fbb6a1-c38c-4208-a63e-69fb6240ebfa None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:56.636477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fbb6a1-c38c-4208-a63e-69fb6240ebfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:56.641684 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05866b25-ad99-4e36-b636-47a27f442d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:56.642332 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05866b25-ad99-4e36-b636-47a27f442d03 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:56.642672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05866b25-ad99-4e36-b636-47a27f442d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:56.642891 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05866b25-ad99-4e36-b636-47a27f442d03 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:56.643833 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fbb6a1-c38c-4208-a63e-69fb6240ebfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:56.645306 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fbb6a1-c38c-4208-a63e-69fb6240ebfa None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:56.653918 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05866b25-ad99-4e36-b636-47a27f442d03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:56.657043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a1fbb6a1-c38c-4208-a63e-69fb6240ebfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:46:56.657043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:56.657043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:56.657043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:56.657043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:56.657043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:56.657043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:56.657043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:56.657043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:56.657043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:56.657043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:56.657043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:56.657043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:46:56.657043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:46:56.657043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:46:56.657915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:46:56.657915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:56.657915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:56.657915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:56.657915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:56.657915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:56.657915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:56.657915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:56.657915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:56.657915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:56.657915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:56.657915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:56.657915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:56.657915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:56.657915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:56.657915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:56.657915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:56.664049 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:46:56.664049 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:46:56.664049 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:46:56.664049 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:46:56.664049 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:46:56.664049 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:46:56.664049 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:46:56.664049 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:56.664049 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 309/631] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:46:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:56.664594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-05866b25-ad99-4e36-b636-47a27f442d03 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:56.664594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:56.664594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:56.664594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:56.664594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:56.664594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:56.664594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:56.664594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:56.664594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:56.664594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:56.664594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:56.664594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:56.664594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:56.664594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:56.664594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:56.664594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:56.665433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:56.665433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:56.665433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:56.665433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:56.665433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:56.665433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:56.665433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:56.665433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:56.665433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:56.665433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:56.665433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:56.665433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:56.665433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:56.665433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:56.665433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:56.665433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:56.666254 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:56.666254 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:56.666254 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:56.666254 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:56.666254 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:56.666254 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:56.666254 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:56.666254 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:56.666254 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:56.666254 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:56.666254 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:56.666254 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:56.666892 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fbb6a1-c38c-4208-a63e-69fb6240ebfa None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:56.666961 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 323/632] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jul 26 04:46:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:56.666961 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ae6c0-3b5e-426c-a07c-299f0354b64c None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:56.666961 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ae6c0-3b5e-426c-a07c-299f0354b64c None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:56.667130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ae6c0-3b5e-426c-a07c-299f0354b64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:56.686452 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-048ae6c0-3b5e-426c-a07c-299f0354b64c None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:56.687763 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-048ae6c0-3b5e-426c-a07c-299f0354b64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:56.733271 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:56.766891 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 324/633] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:56.767823 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ae6c0-3b5e-426c-a07c-299f0354b64c None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:56.780009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:56.792696 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fbb6a1-c38c-4208-a63e-69fb6240ebfa None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:56.838403 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e240e4-fd21-4e8b-91cb-47a3363ba43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:56.840081 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e240e4-fd21-4e8b-91cb-47a3363ba43b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:56.840377 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e240e4-fd21-4e8b-91cb-47a3363ba43b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:56.840663 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e240e4-fd21-4e8b-91cb-47a3363ba43b None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:56.841382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e240e4-fd21-4e8b-91cb-47a3363ba43b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:56.842065 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e240e4-fd21-4e8b-91cb-47a3363ba43b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:56.843500 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e240e4-fd21-4e8b-91cb-47a3363ba43b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:56.855974 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 310/634] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:56.856947 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e240e4-fd21-4e8b-91cb-47a3363ba43b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:56.881143 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ae6c0-3b5e-426c-a07c-299f0354b64c None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:56.917987 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243928d8-55c8-4ffd-a2dd-8a20897341f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:56.918952 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243928d8-55c8-4ffd-a2dd-8a20897341f8 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:56.919372 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243928d8-55c8-4ffd-a2dd-8a20897341f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:56.919813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243928d8-55c8-4ffd-a2dd-8a20897341f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:56.920792 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243928d8-55c8-4ffd-a2dd-8a20897341f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:56.921237 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243928d8-55c8-4ffd-a2dd-8a20897341f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:56.923054 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243928d8-55c8-4ffd-a2dd-8a20897341f8 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:56.935718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 325/635] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:46:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:56.936568 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243928d8-55c8-4ffd-a2dd-8a20897341f8 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:56.967531 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e240e4-fd21-4e8b-91cb-47a3363ba43b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:56.990533 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:57.003871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb78e6-3086-4867-ac13-b9700d8852bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:57.004504 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb78e6-3086-4867-ac13-b9700d8852bc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:57.004786 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb78e6-3086-4867-ac13-b9700d8852bc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:57.005083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb78e6-3086-4867-ac13-b9700d8852bc None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:57.005724 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:57.005844 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb78e6-3086-4867-ac13-b9700d8852bc None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:57.006072 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb78e6-3086-4867-ac13-b9700d8852bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:57.007643 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb78e6-3086-4867-ac13-b9700d8852bc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:57.016365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 311/636] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jul 26 04:46:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:57.017183 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb78e6-3086-4867-ac13-b9700d8852bc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:57.041148 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243928d8-55c8-4ffd-a2dd-8a20897341f8 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:57.075101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2677e81d-4f87-49ea-9e10-e10c85956eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:57.075775 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677e81d-4f87-49ea-9e10-e10c85956eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:57.076055 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677e81d-4f87-49ea-9e10-e10c85956eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:57.076362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677e81d-4f87-49ea-9e10-e10c85956eb2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:57.086754 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2677e81d-4f87-49ea-9e10-e10c85956eb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:57.093475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-2677e81d-4f87-49ea-9e10-e10c85956eb2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:57.093475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:57.093475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:57.093475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:57.093475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:57.093475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:57.093475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:57.093475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:57.093475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:57.093475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:57.093475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:57.093475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:57.093475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:57.093475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:57.093475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:57.093475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:57.094526 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:57.094526 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:57.094526 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:57.094526 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:57.094526 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:57.094526 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:57.094526 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:57.094526 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:57.094526 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:57.094526 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:57.094526 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:57.094526 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:57.094526 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:57.094526 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:57.094526 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:57.094526 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:57.095360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:57.095360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:57.095360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:57.095360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:57.095360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:57.095360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:57.095360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:57.095360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:57.095360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:57.095360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:57.095360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:57.095360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:57.095360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 326/637] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jul 26 04:46:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:57.096513 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347703f-efff-4026-8c31-4625ba793d14 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:57.096735 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347703f-efff-4026-8c31-4625ba793d14 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:57.096915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347703f-efff-4026-8c31-4625ba793d14 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:57.098189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 327/638] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:57.103285 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347703f-efff-4026-8c31-4625ba793d14 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:57.119037 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb78e6-3086-4867-ac13-b9700d8852bc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:57.153226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fae50e-352a-4da5-921b-5b60aea913b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:57.153882 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fae50e-352a-4da5-921b-5b60aea913b8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:57.154167 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fae50e-352a-4da5-921b-5b60aea913b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:57.154618 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fae50e-352a-4da5-921b-5b60aea913b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:57.161524 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fae50e-352a-4da5-921b-5b60aea913b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:57.162983 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fae50e-352a-4da5-921b-5b60aea913b8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:57.174807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-55fae50e-352a-4da5-921b-5b60aea913b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:46:57.174807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:57.174807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:57.174807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:57.174807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:57.174807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:57.174807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:57.174807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:57.174807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:57.174807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:57.174807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:57.174807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:57.174807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:46:57.174807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:46:57.174807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:46:57.175870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:46:57.175870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:57.175870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:57.175870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:57.175870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:57.175870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:57.175870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:57.175870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:57.175870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:57.175870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:57.175870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:57.175870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:57.175870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:57.175870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:57.175870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:57.175870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:57.175870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:57.176740 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:46:57.176740 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:46:57.176740 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:46:57.176740 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:46:57.176740 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:46:57.176740 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:46:57.176740 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:46:57.176740 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:57.176740 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 312/639] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:46:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:57.176740 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fae50e-352a-4da5-921b-5b60aea913b8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:57.206655 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347703f-efff-4026-8c31-4625ba793d14 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:57.222010 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:57.234178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:57.242651 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e241f41-0c23-468e-8fb0-0f5c5b99394b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:57.243333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e241f41-0c23-468e-8fb0-0f5c5b99394b None admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:57.243645 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e241f41-0c23-468e-8fb0-0f5c5b99394b None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:57.243923 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e241f41-0c23-468e-8fb0-0f5c5b99394b None admin] PATH_INFO: `/v3/projects/d583e0be8ef74766bffd6c3c07af18f1/users/d5218319aaa5424882714bffcbeee730/roles/e5048f4e7c6b4494a15d628758090380` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:57.258716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e241f41-0c23-468e-8fb0-0f5c5b99394b None admin] RBAC: Authorizing `identity:create_grant(project_id=d583e0be8ef74766bffd6c3c07af18f1, user_id=d5218319aaa5424882714bffcbeee730, role_id=e5048f4e7c6b4494a15d628758090380)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:57.260364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e241f41-0c23-468e-8fb0-0f5c5b99394b None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:57.273905 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 328/640] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jul 26 04:46:57 2022] PUT /identity/v3/projects/d583e0be8ef74766bffd6c3c07af18f1/users/d5218319aaa5424882714bffcbeee730/roles/e5048f4e7c6b4494a15d628758090380 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:46:57.274727 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e241f41-0c23-468e-8fb0-0f5c5b99394b None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:57.292608 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fae50e-352a-4da5-921b-5b60aea913b8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:57.341274 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0761e6f-60e5-4348-a5b1-7b5a02340205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:57.342333 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0761e6f-60e5-4348-a5b1-7b5a02340205 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:57.342618 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0761e6f-60e5-4348-a5b1-7b5a02340205 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:57.342923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0761e6f-60e5-4348-a5b1-7b5a02340205 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:57.355276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0761e6f-60e5-4348-a5b1-7b5a02340205 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:57.356834 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0761e6f-60e5-4348-a5b1-7b5a02340205 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:57.374125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a0761e6f-60e5-4348-a5b1-7b5a02340205 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:57.374125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:57.374125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:57.374125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:57.374125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:57.374125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:57.374125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:57.374125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:57.374125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:57.374125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:57.374125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:57.374125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:57.374125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:57.374125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:57.374125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:57.375182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:57.375182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:57.375182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:57.375182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:57.375182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:57.375182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:57.375182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:57.375182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:57.375182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:57.375182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:57.375182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:57.375182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:57.375182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:57.375182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:57.375182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:57.375182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:57.375182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:57.376524 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:57.376524 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:57.376524 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:57.376524 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:57.379759 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 313/641] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:57.382401 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f31c3a-4144-4c04-8652-c60925885d10 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:57.382845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f31c3a-4144-4c04-8652-c60925885d10 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:57.383265 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f31c3a-4144-4c04-8652-c60925885d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:57.405498 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-38f31c3a-4144-4c04-8652-c60925885d10 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:57.406034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e241f41-0c23-468e-8fb0-0f5c5b99394b None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:57.406738 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-38f31c3a-4144-4c04-8652-c60925885d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:57.451793 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb34707-55e5-4bf9-91d6-3a2354da4694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:57.452361 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb34707-55e5-4bf9-91d6-3a2354da4694 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:57.452649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb34707-55e5-4bf9-91d6-3a2354da4694 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:57.452925 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb34707-55e5-4bf9-91d6-3a2354da4694 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:57.453679 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb34707-55e5-4bf9-91d6-3a2354da4694 None admin] RBAC: Adding query filter params (name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:57.453984 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb34707-55e5-4bf9-91d6-3a2354da4694 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:57.455391 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb34707-55e5-4bf9-91d6-3a2354da4694 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:57.464794 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 329/642] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jul 26 04:46:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:57.465672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb34707-55e5-4bf9-91d6-3a2354da4694 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:57.493370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 314/643] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:57.494035 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f31c3a-4144-4c04-8652-c60925885d10 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:57.525964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:57.528212 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:57.594881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb34707-55e5-4bf9-91d6-3a2354da4694 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:57.611262 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f31c3a-4144-4c04-8652-c60925885d10 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:57.644150 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf5abc-272d-470f-9557-801ff59e868a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:57.644842 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf5abc-272d-470f-9557-801ff59e868a None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:57.645082 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf5abc-272d-470f-9557-801ff59e868a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:57.645324 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf5abc-272d-470f-9557-801ff59e868a None admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:57.646101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf5abc-272d-470f-9557-801ff59e868a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:57.646413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf5abc-272d-470f-9557-801ff59e868a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:57.647914 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf5abc-272d-470f-9557-801ff59e868a None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:57.655841 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169ebe4-bb4c-40cd-aba1-fdd0d0abde15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:57.656480 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169ebe4-bb4c-40cd-aba1-fdd0d0abde15 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:57.656762 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169ebe4-bb4c-40cd-aba1-fdd0d0abde15 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:57.657052 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169ebe4-bb4c-40cd-aba1-fdd0d0abde15 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:57.657736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169ebe4-bb4c-40cd-aba1-fdd0d0abde15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:57.658035 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169ebe4-bb4c-40cd-aba1-fdd0d0abde15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:57.659435 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169ebe4-bb4c-40cd-aba1-fdd0d0abde15 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:57.662425 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 330/644] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:46:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:57.663242 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf5abc-272d-470f-9557-801ff59e868a None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:57.681169 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 315/645] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:57.681169 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169ebe4-bb4c-40cd-aba1-fdd0d0abde15 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:57.782178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:57.785864 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf5abc-272d-470f-9557-801ff59e868a None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:57.788642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:57.809471 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169ebe4-bb4c-40cd-aba1-fdd0d0abde15 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:57.830595 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b8e83-1399-42b6-a698-5fb2e3725b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:57.831208 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b8e83-1399-42b6-a698-5fb2e3725b7d None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:57.831488 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b8e83-1399-42b6-a698-5fb2e3725b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:57.831806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b8e83-1399-42b6-a698-5fb2e3725b7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:57.844345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b8e83-1399-42b6-a698-5fb2e3725b7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:57.845939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b8e83-1399-42b6-a698-5fb2e3725b7d None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:57.850674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948e65b-dd4f-4a93-9f4a-ade504428fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:57.851524 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948e65b-dd4f-4a93-9f4a-ade504428fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:57.851965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948e65b-dd4f-4a93-9f4a-ade504428fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:57.852372 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948e65b-dd4f-4a93-9f4a-ade504428fc8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:57.863469 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-581b8e83-1399-42b6-a698-5fb2e3725b7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:57.863469 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:57.863469 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:57.863469 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:57.863469 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:57.863469 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:57.863469 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:57.863469 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:57.863469 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:57.863469 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:57.863469 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:57.863469 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:57.863469 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:57.863469 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:57.863469 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:57.864579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:57.864579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:57.864579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:57.864579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:57.864579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:57.864579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:57.864579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:57.864579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:57.864579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:57.864579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:57.864579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:57.864579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:57.864579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:57.864579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:57.864579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:57.864579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:57.864579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:57.865621 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:57.865621 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:57.865621 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:57.865621 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:57.865621 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 331/646] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:57.865621 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b8e83-1399-42b6-a698-5fb2e3725b7d None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:57.866146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5948e65b-dd4f-4a93-9f4a-ade504428fc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:57.871455 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-5948e65b-dd4f-4a93-9f4a-ade504428fc8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:57.871455 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:57.871455 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:57.871455 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:57.871455 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:57.871455 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:57.871455 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:57.871455 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:57.871455 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:57.871455 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:57.871455 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:57.871455 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:57.871455 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:57.871455 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:57.871455 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:57.871455 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:57.872588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:57.872588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:57.872588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:57.872588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:57.872588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:57.872588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:57.872588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:57.872588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:57.872588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:57.872588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:57.872588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:57.872588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:57.872588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:57.872588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:57.872588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:57.872588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:57.873539 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:57.873539 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:57.873539 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:57.873539 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:57.873539 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:57.873539 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:57.873539 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:57.873539 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:57.873539 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:57.873539 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:57.873539 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:57.873539 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:57.877633 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 316/647] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jul 26 04:46:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:57.878441 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948e65b-dd4f-4a93-9f4a-ade504428fc8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:58.013545 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948e65b-dd4f-4a93-9f4a-ade504428fc8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:58.054547 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccccaeb0-3d74-4e9b-8360-8f2c09ae356b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:58.055118 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccaeb0-3d74-4e9b-8360-8f2c09ae356b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:58.055402 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccaeb0-3d74-4e9b-8360-8f2c09ae356b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:58.055720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccaeb0-3d74-4e9b-8360-8f2c09ae356b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:58.066166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccccaeb0-3d74-4e9b-8360-8f2c09ae356b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:58.067971 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccccaeb0-3d74-4e9b-8360-8f2c09ae356b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:58.081674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-ccccaeb0-3d74-4e9b-8360-8f2c09ae356b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 04:46:58.081674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:58.081674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:58.081674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:58.081674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:58.081674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:58.081674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:58.081674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:58.081674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:58.081674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:58.081674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:58.081674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:58.081674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:46:58.081674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:46:58.081674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:46:58.081674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:46:58.083186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:58.083186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:58.083186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:46:58.083186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:58.083186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:46:58.083186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:58.083186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:58.083186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:58.083186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:58.083186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:58.083186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:58.083186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:58.083186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:58.083186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:58.083186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:58.083186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:58.083186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:58.083186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:58.084567 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:58.084567 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:58.084567 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:46:58.084567 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:46:58.084567 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:46:58.084567 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:46:58.084567 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:46:58.084567 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:46:58.084567 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 04:46:58.084567 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:58.084567 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 317/648] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:58.084567 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccccaeb0-3d74-4e9b-8360-8f2c09ae356b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:58.096842 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:58.113576 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b8e83-1399-42b6-a698-5fb2e3725b7d None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:58.150015 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adcc086-cdd7-441e-82c8-5f615c3bd863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:58.151149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adcc086-cdd7-441e-82c8-5f615c3bd863 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:58.151704 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adcc086-cdd7-441e-82c8-5f615c3bd863 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:58.152121 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adcc086-cdd7-441e-82c8-5f615c3bd863 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:58.153053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adcc086-cdd7-441e-82c8-5f615c3bd863 None admin] RBAC: Adding query filter params (role.id=e5048f4e7c6b4494a15d628758090380, scope.project.id=d583e0be8ef74766bffd6c3c07af18f1, user.id=2faeed9002ee4154bc980c146cd06bcc) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:58.153503 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adcc086-cdd7-441e-82c8-5f615c3bd863 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:58.155457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adcc086-cdd7-441e-82c8-5f615c3bd863 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:58.162597 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 332/649] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:46:57 2022] GET /identity/v3/role_assignments?user.id=2faeed9002ee4154bc980c146cd06bcc&scope.project.id=d583e0be8ef74766bffd6c3c07af18f1&role.id=e5048f4e7c6b4494a15d628758090380 => generated 608 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:58.163749 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adcc086-cdd7-441e-82c8-5f615c3bd863 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:58.185439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:58.237358 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccccaeb0-3d74-4e9b-8360-8f2c09ae356b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:58.279673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e6160-6020-451a-8ea1-609473c0fd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:58.280330 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e6160-6020-451a-8ea1-609473c0fd8b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:58.280611 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e6160-6020-451a-8ea1-609473c0fd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:58.280889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e6160-6020-451a-8ea1-609473c0fd8b None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:58.281745 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e6160-6020-451a-8ea1-609473c0fd8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:58.281981 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e6160-6020-451a-8ea1-609473c0fd8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:58.283422 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e6160-6020-451a-8ea1-609473c0fd8b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:58.297203 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adcc086-cdd7-441e-82c8-5f615c3bd863 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:58.299283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 318/650] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:58.301444 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e6160-6020-451a-8ea1-609473c0fd8b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:58.340876 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc709f9-a46f-46b5-beab-9a1f352ad0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:58.341461 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc709f9-a46f-46b5-beab-9a1f352ad0ff None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:58.342104 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc709f9-a46f-46b5-beab-9a1f352ad0ff None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:58.342379 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc709f9-a46f-46b5-beab-9a1f352ad0ff None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:58.343145 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc709f9-a46f-46b5-beab-9a1f352ad0ff None admin] RBAC: Adding query filter params (name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:58.343452 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc709f9-a46f-46b5-beab-9a1f352ad0ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:58.344964 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc709f9-a46f-46b5-beab-9a1f352ad0ff None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:58.359611 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 333/651] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jul 26 04:46:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:58.360694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc709f9-a46f-46b5-beab-9a1f352ad0ff None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:58.403411 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:58.463784 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:58.465605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e6160-6020-451a-8ea1-609473c0fd8b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:58.501639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc709f9-a46f-46b5-beab-9a1f352ad0ff None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:58.516947 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5a353-45ef-470e-b5f1-5e5293f16fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:58.517598 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5a353-45ef-470e-b5f1-5e5293f16fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:58.517994 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5a353-45ef-470e-b5f1-5e5293f16fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:58.518268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5a353-45ef-470e-b5f1-5e5293f16fd4 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:58.519195 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5a353-45ef-470e-b5f1-5e5293f16fd4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:58.519500 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5a353-45ef-470e-b5f1-5e5293f16fd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:58.521319 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5a353-45ef-470e-b5f1-5e5293f16fd4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:58.542270 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982592c7-a27d-469b-81ce-5b9651f8faac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:58.543077 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982592c7-a27d-469b-81ce-5b9651f8faac None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:58.543482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982592c7-a27d-469b-81ce-5b9651f8faac None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:58.543639 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 319/652] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Jul 26 04:46:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:58.543886 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982592c7-a27d-469b-81ce-5b9651f8faac None admin] PATH_INFO: `/v3/users/glance` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:58.544417 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5a353-45ef-470e-b5f1-5e5293f16fd4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:58.552796 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982592c7-a27d-469b-81ce-5b9651f8faac None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:58.554401 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982592c7-a27d-469b-81ce-5b9651f8faac None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:58.568582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-982592c7-a27d-469b-81ce-5b9651f8faac None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 26 04:46:58.568582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:58.568582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:58.568582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:58.568582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:58.568582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:58.568582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:58.568582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:58.568582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:58.568582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:58.568582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:58.568582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:58.568582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:46:58.568582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:46:58.568582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:46:58.568582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:46:58.569602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:58.569602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:58.569602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:46:58.569602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:58.569602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:46:58.569602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:58.569602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:58.569602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:58.569602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:58.569602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:58.569602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:58.569602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:58.569602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:58.569602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:58.569602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:58.569602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:58.569602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:58.569602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:58.570799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:58.570799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:58.570799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:46:58.570799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:46:58.570799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:46:58.570799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:46:58.570799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:46:58.570799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:46:58.570799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 26 04:46:58.570799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:58.570799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 334/653] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:46:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:58.571403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982592c7-a27d-469b-81ce-5b9651f8faac None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:58.680541 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5a353-45ef-470e-b5f1-5e5293f16fd4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:58.694977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982592c7-a27d-469b-81ce-5b9651f8faac None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:58.711230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:58.724393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75737e-ea79-42f1-b32a-62bf5b8d567c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:58.725525 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75737e-ea79-42f1-b32a-62bf5b8d567c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:58.725876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75737e-ea79-42f1-b32a-62bf5b8d567c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:58.726217 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75737e-ea79-42f1-b32a-62bf5b8d567c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:58.737765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc112c1b-0742-488a-bcf0-004a0dcd5d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:58.738726 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc112c1b-0742-488a-bcf0-004a0dcd5d45 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:58.739420 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc112c1b-0742-488a-bcf0-004a0dcd5d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:58.739896 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc112c1b-0742-488a-bcf0-004a0dcd5d45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:58.743513 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75737e-ea79-42f1-b32a-62bf5b8d567c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:58.746134 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75737e-ea79-42f1-b32a-62bf5b8d567c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:58.750936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:58.763398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-ba75737e-ea79-42f1-b32a-62bf5b8d567c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:58.763398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:58.763398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:58.763398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:58.763398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:58.763398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:58.763398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:58.763398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:58.763398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:58.763398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:58.763398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:58.763398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:58.763398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:58.763398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:58.763398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:58.764395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:58.764395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:58.764395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:58.764395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:58.764395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:58.764395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:58.764395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:58.764395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:58.764395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:58.764395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:58.764395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:58.764395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:58.764395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:58.764395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:58.764395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:58.764395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:58.764395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:58.765297 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:58.765297 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:58.765297 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:58.765297 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:58.772004 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 320/654] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:58.773149 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75737e-ea79-42f1-b32a-62bf5b8d567c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:58.829542 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc112c1b-0742-488a-bcf0-004a0dcd5d45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:58.831452 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc112c1b-0742-488a-bcf0-004a0dcd5d45 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:58.880497 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75737e-ea79-42f1-b32a-62bf5b8d567c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:58.928284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230aeda0-2b28-422a-ba1b-b0254128f3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:58.929093 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230aeda0-2b28-422a-ba1b-b0254128f3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:58.929464 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230aeda0-2b28-422a-ba1b-b0254128f3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:58.929832 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230aeda0-2b28-422a-ba1b-b0254128f3d6 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:58.930789 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230aeda0-2b28-422a-ba1b-b0254128f3d6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:58.931163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230aeda0-2b28-422a-ba1b-b0254128f3d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:58.933386 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230aeda0-2b28-422a-ba1b-b0254128f3d6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:58.959129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 321/655] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Jul 26 04:46:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:58.960145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230aeda0-2b28-422a-ba1b-b0254128f3d6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:58.965047 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 335/656] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:46:58.966300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc112c1b-0742-488a-bcf0-004a0dcd5d45 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:59.007283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:59.011732 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:59.083184 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230aeda0-2b28-422a-ba1b-b0254128f3d6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:59.098792 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc112c1b-0742-488a-bcf0-004a0dcd5d45 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:59.129092 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55b7ed-d763-4e7a-9f50-4d1accfb0b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:59.129715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55b7ed-d763-4e7a-9f50-4d1accfb0b68 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:59.130001 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55b7ed-d763-4e7a-9f50-4d1accfb0b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:59.130279 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55b7ed-d763-4e7a-9f50-4d1accfb0b68 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:59.130981 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f55b7ed-d763-4e7a-9f50-4d1accfb0b68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:59.131288 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f55b7ed-d763-4e7a-9f50-4d1accfb0b68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:59.132944 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f55b7ed-d763-4e7a-9f50-4d1accfb0b68 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:59.145913 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033735c6-1aa4-414e-818d-dc2ab929df52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:59.147288 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033735c6-1aa4-414e-818d-dc2ab929df52 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:59.147850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033735c6-1aa4-414e-818d-dc2ab929df52 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:59.148330 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033735c6-1aa4-414e-818d-dc2ab929df52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:59.150334 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 322/657] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:59.151310 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55b7ed-d763-4e7a-9f50-4d1accfb0b68 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:59.241728 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033735c6-1aa4-414e-818d-dc2ab929df52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:59.244240 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033735c6-1aa4-414e-818d-dc2ab929df52 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:59.255105 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:59.263459 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55b7ed-d763-4e7a-9f50-4d1accfb0b68 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:59.298293 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:59.301075 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8f17e-9dd2-4e46-8d20-867b22d7e88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:59.301793 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8f17e-9dd2-4e46-8d20-867b22d7e88e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:59.302075 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8f17e-9dd2-4e46-8d20-867b22d7e88e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:59.302463 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8f17e-9dd2-4e46-8d20-867b22d7e88e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:59.315739 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa8f17e-9dd2-4e46-8d20-867b22d7e88e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:59.317319 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa8f17e-9dd2-4e46-8d20-867b22d7e88e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:59.336007 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-caa8f17e-9dd2-4e46-8d20-867b22d7e88e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:59.336007 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:59.336007 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:59.336007 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:59.336007 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:59.336007 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:59.336007 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:59.336007 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:59.336007 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:59.336007 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:59.336007 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:59.336007 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:59.336007 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:59.336007 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:59.336007 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:59.336965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:59.336965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:59.336965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:59.336965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:59.336965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:59.336965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:59.336965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:59.336965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:59.336965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:59.336965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:59.336965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:59.336965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:59.336965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:59.336965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:59.336965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:59.336965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:59.336965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:59.337862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:59.337862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:59.337862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:59.337862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:59.339740 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 323/658] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:59.340949 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8f17e-9dd2-4e46-8d20-867b22d7e88e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:59.371686 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 336/659] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:46:59.372803 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa6f97-aa78-4c1a-83bb-344459e938ad None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:59.373150 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa6f97-aa78-4c1a-83bb-344459e938ad None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:59.373358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa6f97-aa78-4c1a-83bb-344459e938ad None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:59.374765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 337/660] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:46:59.375285 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa6f97-aa78-4c1a-83bb-344459e938ad None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:59.462585 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8f17e-9dd2-4e46-8d20-867b22d7e88e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:59.490200 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa6f97-aa78-4c1a-83bb-344459e938ad None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:59.499986 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63ff64-7a03-475a-93c5-820cca15fa53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:59.500658 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63ff64-7a03-475a-93c5-820cca15fa53 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:59.500833 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63ff64-7a03-475a-93c5-820cca15fa53 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:59.501110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63ff64-7a03-475a-93c5-820cca15fa53 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:59.510062 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63ff64-7a03-475a-93c5-820cca15fa53 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:59.511755 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63ff64-7a03-475a-93c5-820cca15fa53 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:59.525002 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-fc63ff64-7a03-475a-93c5-820cca15fa53 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 04:46:59.525002 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:59.525002 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:59.525002 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:59.525002 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:59.525002 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:59.525002 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:59.525002 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:59.525002 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:59.525002 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:59.525002 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:59.525002 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:59.525002 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:46:59.525002 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:46:59.525002 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:46:59.525002 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:46:59.526110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:59.526110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:59.526110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:46:59.526110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:59.526110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:46:59.526110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:46:59.526110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:59.526110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:59.526110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:59.526110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:59.526110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:59.526110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:59.526110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:59.526110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:59.526110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:59.526110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:59.526110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:59.526110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:59.527076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:59.527076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:59.527076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:46:59.527076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:46:59.527076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:46:59.527076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:46:59.527076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:46:59.527076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:46:59.527076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 04:46:59.527076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:59.527076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 324/661] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jul 26 04:46:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:59.527777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b731e6-fc71-46f1-a447-bd0012a8b6ac None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:59.528962 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b731e6-fc71-46f1-a447-bd0012a8b6ac None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:59.528962 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b731e6-fc71-46f1-a447-bd0012a8b6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:59.541271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dee7ca-1d5b-4558-bf5b-2b29f0ce0bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:59.541894 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:59.542403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dee7ca-1d5b-4558-bf5b-2b29f0ce0bdb None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:59.542728 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dee7ca-1d5b-4558-bf5b-2b29f0ce0bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:59.543010 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dee7ca-1d5b-4558-bf5b-2b29f0ce0bdb None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:59.544072 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dee7ca-1d5b-4558-bf5b-2b29f0ce0bdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:59.544426 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dee7ca-1d5b-4558-bf5b-2b29f0ce0bdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:59.545805 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dee7ca-1d5b-4558-bf5b-2b29f0ce0bdb None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:59.548135 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-43b731e6-fc71-46f1-a447-bd0012a8b6ac None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:46:59.549349 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-43b731e6-fc71-46f1-a447-bd0012a8b6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:46:59.554018 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:59.559959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 338/662] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:59.560835 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dee7ca-1d5b-4558-bf5b-2b29f0ce0bdb None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:59.628058 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 325/663] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:46:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:46:59.628931 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b731e6-fc71-46f1-a447-bd0012a8b6ac None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:59.682457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dee7ca-1d5b-4558-bf5b-2b29f0ce0bdb None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:59.719833 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6638b-ad91-4a40-b8a0-53cb23c23b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:59.720482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6638b-ad91-4a40-b8a0-53cb23c23b0d None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:59.720761 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6638b-ad91-4a40-b8a0-53cb23c23b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:59.721037 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6638b-ad91-4a40-b8a0-53cb23c23b0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:59.731249 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6638b-ad91-4a40-b8a0-53cb23c23b0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:59.732746 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6638b-ad91-4a40-b8a0-53cb23c23b0d None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:59.740375 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b731e6-fc71-46f1-a447-bd0012a8b6ac None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:59.747270 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-66a6638b-ad91-4a40-b8a0-53cb23c23b0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:59.747270 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:59.747270 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:59.747270 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:59.747270 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:59.747270 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:59.747270 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:59.747270 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:59.747270 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:59.747270 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:59.747270 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:59.747270 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:59.747270 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:46:59.747270 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:46:59.747270 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:46:59.748298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:46:59.748298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:59.748298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:59.748298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:59.748298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:59.748298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:59.748298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:59.748298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:59.748298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:59.748298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:59.748298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:59.748298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:59.748298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:59.748298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:59.748298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:59.748298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:59.748298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:59.749230 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:46:59.749230 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:46:59.749230 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:46:59.749230 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:59.749230 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 339/664] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:46:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:59.749522 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6638b-ad91-4a40-b8a0-53cb23c23b0d None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:59.781830 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e0b78-2a89-47f5-842f-92f88d8d040c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:59.782377 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e0b78-2a89-47f5-842f-92f88d8d040c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:59.782656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e0b78-2a89-47f5-842f-92f88d8d040c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:59.782927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e0b78-2a89-47f5-842f-92f88d8d040c None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:59.783760 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e0b78-2a89-47f5-842f-92f88d8d040c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:46:59.784069 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e0b78-2a89-47f5-842f-92f88d8d040c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:59.785502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e0b78-2a89-47f5-842f-92f88d8d040c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:59.788923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:59.805226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 326/665] 10.222.0.17 () {56 vars in 1139 bytes} [Tue Jul 26 04:46:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:46:59.807303 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:46:59.808908 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e0b78-2a89-47f5-842f-92f88d8d040c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:59.862794 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6638b-ad91-4a40-b8a0-53cb23c23b0d None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:59.900971 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5195b-7e2c-49f4-96e6-8f9874139a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:59.901632 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5195b-7e2c-49f4-96e6-8f9874139a99 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:59.902019 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5195b-7e2c-49f4-96e6-8f9874139a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:59.902300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5195b-7e2c-49f4-96e6-8f9874139a99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:59.911688 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d5195b-7e2c-49f4-96e6-8f9874139a99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:46:59.913162 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d5195b-7e2c-49f4-96e6-8f9874139a99 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:46:59.913288 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e0b78-2a89-47f5-842f-92f88d8d040c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:46:59.928168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-84d5195b-7e2c-49f4-96e6-8f9874139a99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:46:59.928168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:59.928168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:59.928168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:59.928168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:59.928168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:59.928168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:59.928168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:59.928168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:59.928168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:59.928168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:59.928168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:59.928168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:46:59.928168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:46:59.928168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:46:59.929108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:46:59.929108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:59.929108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:59.929108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:59.929108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:59.929108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:59.929108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:59.929108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:59.929108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:59.929108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:59.929108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:59.929108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:59.929108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:59.929108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:59.929108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:59.929108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:59.929108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:59.930002 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:46:59.930002 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:46:59.930002 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:46:59.930002 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:46:59.930002 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:46:59.930002 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:46:59.930002 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:46:59.930002 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:46:59.931903 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 340/666] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:46:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:59.932527 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5195b-7e2c-49f4-96e6-8f9874139a99 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:46:59.951360 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4243953d-313b-417e-85df-6d2f5b526c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:46:59.952070 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243953d-313b-417e-85df-6d2f5b526c07 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:46:59.952353 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243953d-313b-417e-85df-6d2f5b526c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:46:59.952629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243953d-313b-417e-85df-6d2f5b526c07 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:46:59.963421 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4243953d-313b-417e-85df-6d2f5b526c07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:46:59.968801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-4243953d-313b-417e-85df-6d2f5b526c07 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:59.968801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:46:59.968801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:46:59.968801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:46:59.968801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:46:59.968801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:46:59.968801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:46:59.968801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:46:59.968801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:46:59.968801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:46:59.968801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:46:59.968801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:46:59.968801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:46:59.968801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:46:59.968801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:46:59.968801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:46:59.969920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:46:59.969920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:46:59.969920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:46:59.969920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:46:59.969920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:46:59.969920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:46:59.969920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:46:59.969920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:46:59.969920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:46:59.969920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:46:59.969920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:46:59.969920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:46:59.969920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:46:59.969920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:46:59.969920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:46:59.969920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:46:59.970770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:46:59.970770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:46:59.970770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:46:59.970770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:46:59.970770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:46:59.970770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:46:59.970770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:46:59.970770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:46:59.970770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:46:59.970770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:46:59.970770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:46:59.970770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:46:59.970770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 327/667] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jul 26 04:46:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:46:59.971451 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4243953d-313b-417e-85df-6d2f5b526c07 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:00.033198 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:00.050389 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5195b-7e2c-49f4-96e6-8f9874139a99 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:00.063519 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:00.081004 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4243953d-313b-417e-85df-6d2f5b526c07 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:00.090459 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2e779-7bb8-41d9-b316-78eead650f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:00.091189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2e779-7bb8-41d9-b316-78eead650f1d None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:00.091922 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2e779-7bb8-41d9-b316-78eead650f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:00.092259 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2e779-7bb8-41d9-b316-78eead650f1d None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:00.093038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2e779-7bb8-41d9-b316-78eead650f1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:00.093416 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2e779-7bb8-41d9-b316-78eead650f1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:00.094893 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2e779-7bb8-41d9-b316-78eead650f1d None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:00.115813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 341/668] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:00.115813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2e779-7bb8-41d9-b316-78eead650f1d None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:00.134797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bca1c2-b807-4705-be8d-bebfbe19b50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:00.135574 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bca1c2-b807-4705-be8d-bebfbe19b50f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:00.135927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bca1c2-b807-4705-be8d-bebfbe19b50f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:00.136347 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bca1c2-b807-4705-be8d-bebfbe19b50f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:00.213923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bca1c2-b807-4705-be8d-bebfbe19b50f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:00.216100 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bca1c2-b807-4705-be8d-bebfbe19b50f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:00.224722 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2e779-7bb8-41d9-b316-78eead650f1d None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:00.263106 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834b6ba-19e5-4b49-9660-5a958e95a004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:00.263866 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834b6ba-19e5-4b49-9660-5a958e95a004 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:00.264219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834b6ba-19e5-4b49-9660-5a958e95a004 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:00.264565 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834b6ba-19e5-4b49-9660-5a958e95a004 None admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:00.265405 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f834b6ba-19e5-4b49-9660-5a958e95a004 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:00.265781 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f834b6ba-19e5-4b49-9660-5a958e95a004 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:00.267316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f834b6ba-19e5-4b49-9660-5a958e95a004 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:00.276706 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:00.284861 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 342/669] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:47:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:00.284861 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834b6ba-19e5-4b49-9660-5a958e95a004 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:00.316624 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:00.344200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 328/670] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:46:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:47:00.345093 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bca1c2-b807-4705-be8d-bebfbe19b50f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:00.405578 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834b6ba-19e5-4b49-9660-5a958e95a004 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:00.441673 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eefe963-eb6f-45ca-a957-701165683959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:00.442586 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eefe963-eb6f-45ca-a957-701165683959 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:00.442883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eefe963-eb6f-45ca-a957-701165683959 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:00.443139 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eefe963-eb6f-45ca-a957-701165683959 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:00.443907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eefe963-eb6f-45ca-a957-701165683959 None admin] RBAC: Adding query filter params (name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:00.444207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eefe963-eb6f-45ca-a957-701165683959 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:00.445594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eefe963-eb6f-45ca-a957-701165683959 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:00.450360 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bca1c2-b807-4705-be8d-bebfbe19b50f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:00.454426 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 343/671] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jul 26 04:47:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:00.455238 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eefe963-eb6f-45ca-a957-701165683959 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:00.486052 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aac016-d056-447d-bf9e-b70b28d6b3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:00.486707 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aac016-d056-447d-bf9e-b70b28d6b3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:00.486987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aac016-d056-447d-bf9e-b70b28d6b3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:00.487264 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aac016-d056-447d-bf9e-b70b28d6b3b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:00.494118 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aac016-d056-447d-bf9e-b70b28d6b3b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:00.495702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aac016-d056-447d-bf9e-b70b28d6b3b6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:00.507594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-f7aac016-d056-447d-bf9e-b70b28d6b3b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:00.507594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:00.507594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:00.507594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:00.507594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:00.507594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:00.507594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:00.507594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:00.507594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:00.507594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:00.507594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:00.507594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:00.507594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:47:00.507594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:47:00.507594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:47:00.508733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:47:00.508733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:00.508733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:00.508733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:00.508733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:00.508733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:00.508733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:00.508733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:00.508733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:00.508733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:00.508733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:00.508733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:00.508733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:00.508733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:00.508733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:00.508733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:00.508733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:00.510042 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:47:00.510042 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:47:00.510042 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:47:00.510042 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:47:00.510042 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:47:00.510042 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:47:00.510042 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:00.510042 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:00.510042 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 329/672] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:47:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:00.510042 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aac016-d056-447d-bf9e-b70b28d6b3b6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:00.526408 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:00.557765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:00.566832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eefe963-eb6f-45ca-a957-701165683959 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:00.603296 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79dd5ef-6810-46b3-a6d4-a36cea581870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:00.603968 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79dd5ef-6810-46b3-a6d4-a36cea581870 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:00.604252 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79dd5ef-6810-46b3-a6d4-a36cea581870 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:00.604521 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79dd5ef-6810-46b3-a6d4-a36cea581870 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:00.605375 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79dd5ef-6810-46b3-a6d4-a36cea581870 None admin] RBAC: Adding query filter params (role.id=e5048f4e7c6b4494a15d628758090380, scope.project.id=d583e0be8ef74766bffd6c3c07af18f1, user.id=79ef3d83ac454fac9f890374cd22507a) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:00.605700 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79dd5ef-6810-46b3-a6d4-a36cea581870 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:00.607162 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79dd5ef-6810-46b3-a6d4-a36cea581870 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:00.613390 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 344/673] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:47:00 2022] GET /identity/v3/role_assignments?user.id=79ef3d83ac454fac9f890374cd22507a&scope.project.id=d583e0be8ef74766bffd6c3c07af18f1&role.id=e5048f4e7c6b4494a15d628758090380 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:00.614622 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8664de8-6f81-4218-998d-ba7a16978b97 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:00.614888 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8664de8-6f81-4218-998d-ba7a16978b97 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:00.615250 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8664de8-6f81-4218-998d-ba7a16978b97 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:00.616692 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 345/674] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:00.617291 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8664de8-6f81-4218-998d-ba7a16978b97 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:00.623603 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aac016-d056-447d-bf9e-b70b28d6b3b6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:00.666043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc13b3-c74d-41ab-8d93-eb7c22b0c13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:00.666727 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc13b3-c74d-41ab-8d93-eb7c22b0c13c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:00.667219 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc13b3-c74d-41ab-8d93-eb7c22b0c13c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:00.667219 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc13b3-c74d-41ab-8d93-eb7c22b0c13c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:00.682679 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc13b3-c74d-41ab-8d93-eb7c22b0c13c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:00.684337 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc13b3-c74d-41ab-8d93-eb7c22b0c13c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:00.707070 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-f4bc13b3-c74d-41ab-8d93-eb7c22b0c13c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:00.707070 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:00.707070 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:00.707070 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:00.707070 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:00.707070 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:00.707070 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:00.707070 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:00.707070 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:00.707070 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:00.707070 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:00.707070 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:00.707070 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:00.707070 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:00.707070 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:00.708549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:00.708549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:00.708549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:00.708549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:00.708549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:00.708549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:00.708549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:00.708549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:00.708549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:00.708549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:00.708549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:00.708549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:00.708549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:00.708549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:00.708549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:00.708549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:00.708549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:00.709453 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:00.709453 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:00.709453 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:00.709453 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:00.710356 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 330/675] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:00.711351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc13b3-c74d-41ab-8d93-eb7c22b0c13c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:00.767545 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8664de8-6f81-4218-998d-ba7a16978b97 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:00.791142 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:00.814490 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac83176c-0a86-4227-93d5-f153a97f2ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:00.815395 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83176c-0a86-4227-93d5-f153a97f2ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:00.815765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83176c-0a86-4227-93d5-f153a97f2ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:00.816048 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83176c-0a86-4227-93d5-f153a97f2ed7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:00.829612 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac83176c-0a86-4227-93d5-f153a97f2ed7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:00.831188 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac83176c-0a86-4227-93d5-f153a97f2ed7 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:00.832602 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc13b3-c74d-41ab-8d93-eb7c22b0c13c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:00.853760 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-ac83176c-0a86-4227-93d5-f153a97f2ed7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:00.853760 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:00.853760 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:00.853760 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:00.853760 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:00.853760 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:00.853760 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:00.853760 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:00.853760 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:00.853760 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:00.853760 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:00.853760 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:00.853760 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:00.853760 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:00.853760 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:00.854827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:00.854827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:00.854827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:00.854827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:00.854827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:00.854827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:00.854827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:00.854827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:00.854827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:00.854827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:00.854827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:00.854827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:00.854827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:00.854827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:00.854827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:00.854827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:00.854827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:00.855873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:00.855873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:00.855873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:00.855873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:00.855873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 346/676] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:00.856921 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff583db7-7632-4d89-b6e8-9261b60fd991 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:00.857354 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff583db7-7632-4d89-b6e8-9261b60fd991 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:00.857468 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff583db7-7632-4d89-b6e8-9261b60fd991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:00.872330 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb0c01-0d81-4b8f-b256-a6d91b1ab098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:00.872970 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb0c01-0d81-4b8f-b256-a6d91b1ab098 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:00.873253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb0c01-0d81-4b8f-b256-a6d91b1ab098 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:00.873531 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb0c01-0d81-4b8f-b256-a6d91b1ab098 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:00.874290 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb0c01-0d81-4b8f-b256-a6d91b1ab098 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:00.874597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb0c01-0d81-4b8f-b256-a6d91b1ab098 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:00.876109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb0c01-0d81-4b8f-b256-a6d91b1ab098 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:00.877570 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-ff583db7-7632-4d89-b6e8-9261b60fd991 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:00.879023 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-ff583db7-7632-4d89-b6e8-9261b60fd991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:00.881429 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:00.890037 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 331/677] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:47:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:00.891255 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb0c01-0d81-4b8f-b256-a6d91b1ab098 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:00.958876 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 347/678] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:47:00.959807 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff583db7-7632-4d89-b6e8-9261b60fd991 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:01.012214 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb0c01-0d81-4b8f-b256-a6d91b1ab098 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:01.041322 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:01.053227 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea15df-e36e-47cc-8978-11b4af11150c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:01.053902 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea15df-e36e-47cc-8978-11b4af11150c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:01.054275 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea15df-e36e-47cc-8978-11b4af11150c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:01.054559 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea15df-e36e-47cc-8978-11b4af11150c None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:01.055309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea15df-e36e-47cc-8978-11b4af11150c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:01.055710 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea15df-e36e-47cc-8978-11b4af11150c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:01.057147 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea15df-e36e-47cc-8978-11b4af11150c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:01.072287 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 332/679] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:01.073089 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea15df-e36e-47cc-8978-11b4af11150c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:01.098251 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff583db7-7632-4d89-b6e8-9261b60fd991 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:01.154250 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32916b9a-8b9e-422c-b446-b1e283d7a37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:01.154916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32916b9a-8b9e-422c-b446-b1e283d7a37b None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:01.155198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32916b9a-8b9e-422c-b446-b1e283d7a37b None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:01.155465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32916b9a-8b9e-422c-b446-b1e283d7a37b None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:01.156203 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32916b9a-8b9e-422c-b446-b1e283d7a37b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:01.156504 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32916b9a-8b9e-422c-b446-b1e283d7a37b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:01.157891 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32916b9a-8b9e-422c-b446-b1e283d7a37b None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:01.169822 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:01.180125 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 348/680] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:01.181063 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32916b9a-8b9e-422c-b446-b1e283d7a37b None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:01.210166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea15df-e36e-47cc-8978-11b4af11150c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:01.246071 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ca7e3-f63e-4311-9754-d58ec01218b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:01.246758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ca7e3-f63e-4311-9754-d58ec01218b1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:01.247042 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ca7e3-f63e-4311-9754-d58ec01218b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:01.247346 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ca7e3-f63e-4311-9754-d58ec01218b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:01.248171 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ca7e3-f63e-4311-9754-d58ec01218b1 None admin] RBAC: Adding query filter params (role.id=e5048f4e7c6b4494a15d628758090380, scope.project.id=d583e0be8ef74766bffd6c3c07af18f1, user.id=6946a788d2ef49ff873ca8e81990527e) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:01.248482 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ca7e3-f63e-4311-9754-d58ec01218b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:01.249924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ca7e3-f63e-4311-9754-d58ec01218b1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:01.267595 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 333/681] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:47:01 2022] GET /identity/v3/role_assignments?user.id=6946a788d2ef49ff873ca8e81990527e&scope.project.id=d583e0be8ef74766bffd6c3c07af18f1&role.id=e5048f4e7c6b4494a15d628758090380 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:01.267595 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ca7e3-f63e-4311-9754-d58ec01218b1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:01.323901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32916b9a-8b9e-422c-b446-b1e283d7a37b None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:01.381876 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907bf384-66d8-4cf0-8042-6472ac965d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:01.382663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bf384-66d8-4cf0-8042-6472ac965d24 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:01.383224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bf384-66d8-4cf0-8042-6472ac965d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:01.385330 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bf384-66d8-4cf0-8042-6472ac965d24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:01.388126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:01.414908 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907bf384-66d8-4cf0-8042-6472ac965d24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:47:01.429263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-907bf384-66d8-4cf0-8042-6472ac965d24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:47:01.429263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:01.429263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:01.429263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:01.429263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:01.429263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:01.429263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:01.429263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:01.429263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:01.429263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:01.429263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:01.429263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:01.429263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:47:01.429263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:47:01.429263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:47:01.429263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:47:01.431075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:47:01.431075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:47:01.431075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:47:01.431075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:47:01.431075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:01.431075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:01.431075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:01.431075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:01.431075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:01.431075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:01.431075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:01.431075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:01.431075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:01.431075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:01.431075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:01.431075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:01.433107 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:01.433107 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:01.433107 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:01.433107 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:01.433107 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:47:01.433107 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:47:01.433107 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:47:01.433107 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:47:01.433107 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:47:01.433107 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:47:01.433107 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:47:01.433107 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:01.439985 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 349/682] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:47:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:01.439985 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907bf384-66d8-4cf0-8042-6472ac965d24 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:01.484394 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ca7e3-f63e-4311-9754-d58ec01218b1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:01.505518 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:01.533200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e41b8-c919-47ab-8154-964a13c14c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:01.533869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e41b8-c919-47ab-8154-964a13c14c85 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:01.534153 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e41b8-c919-47ab-8154-964a13c14c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:01.534425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e41b8-c919-47ab-8154-964a13c14c85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:01.544385 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e41b8-c919-47ab-8154-964a13c14c85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:01.545960 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e41b8-c919-47ab-8154-964a13c14c85 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:01.557167 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907bf384-66d8-4cf0-8042-6472ac965d24 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:01.558782 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-885e41b8-c919-47ab-8154-964a13c14c85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:01.558782 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:01.558782 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:01.558782 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:01.558782 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:01.558782 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:01.558782 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:01.558782 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:01.558782 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:01.558782 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:01.558782 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:01.558782 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:01.558782 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:47:01.558782 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:47:01.558782 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:47:01.559889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:47:01.559889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:01.559889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:01.559889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:01.559889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:01.559889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:01.559889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:01.559889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:01.559889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:01.559889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:01.559889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:01.559889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:01.559889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:01.559889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:01.559889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:01.559889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:01.559889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:01.561025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:47:01.561025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:47:01.561025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:47:01.561025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:47:01.561025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:47:01.561025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:47:01.561025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:01.561025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:01.566701 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 334/683] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:47:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:01.566701 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e41b8-c919-47ab-8154-964a13c14c85 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:01.598468 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7411c-7076-4924-ac80-aa4364eef83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:01.599132 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7411c-7076-4924-ac80-aa4364eef83c None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:01.599477 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7411c-7076-4924-ac80-aa4364eef83c None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:01.599787 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7411c-7076-4924-ac80-aa4364eef83c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:01.608907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f7411c-7076-4924-ac80-aa4364eef83c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:01.610378 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f7411c-7076-4924-ac80-aa4364eef83c None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:01.622539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-23f7411c-7076-4924-ac80-aa4364eef83c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 04:47:01.622539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:01.622539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:01.622539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:01.622539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:01.622539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:01.622539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:01.622539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:01.622539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:01.622539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:01.622539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:01.622539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:01.622539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:47:01.622539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:47:01.622539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:47:01.622539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:47:01.623800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:01.623800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:01.623800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:47:01.623800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:01.623800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:47:01.623800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:01.623800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:01.623800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:01.623800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:01.623800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:01.623800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:01.623800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:01.623800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:01.623800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:01.623800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:01.623800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:01.623800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:01.623800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:01.624920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:01.624920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:01.624920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:47:01.624920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:47:01.624920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:47:01.624920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:47:01.624920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:47:01.624920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:47:01.624920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 04:47:01.624920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:01.624920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 350/684] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:47:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:01.625509 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7411c-7076-4924-ac80-aa4364eef83c None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:01.697338 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e41b8-c919-47ab-8154-964a13c14c85 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:01.709113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:01.735964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd9701-14d5-41a1-946b-dd3d7524a78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:01.736637 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd9701-14d5-41a1-946b-dd3d7524a78c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:01.737176 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd9701-14d5-41a1-946b-dd3d7524a78c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:01.737319 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd9701-14d5-41a1-946b-dd3d7524a78c None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:01.737944 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd9701-14d5-41a1-946b-dd3d7524a78c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:01.738249 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd9701-14d5-41a1-946b-dd3d7524a78c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:01.739803 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd9701-14d5-41a1-946b-dd3d7524a78c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:01.749408 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 335/685] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:47:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:01.750533 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd9701-14d5-41a1-946b-dd3d7524a78c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:01.765059 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7411c-7076-4924-ac80-aa4364eef83c None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:01.788566 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:01.808378 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c058b-b2a9-4d9e-8a81-23892fbe03f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:01.809183 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c058b-b2a9-4d9e-8a81-23892fbe03f8 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:01.809574 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c058b-b2a9-4d9e-8a81-23892fbe03f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:01.809973 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c058b-b2a9-4d9e-8a81-23892fbe03f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:01.810851 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c058b-b2a9-4d9e-8a81-23892fbe03f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:01.811263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c058b-b2a9-4d9e-8a81-23892fbe03f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:01.813011 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c058b-b2a9-4d9e-8a81-23892fbe03f8 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:01.833748 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 351/686] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:47:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:01.834814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c058b-b2a9-4d9e-8a81-23892fbe03f8 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:01.911953 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd9701-14d5-41a1-946b-dd3d7524a78c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:01.957494 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec73c79-2d3f-4dca-8511-4ed40c55aa06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:01.957942 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c058b-b2a9-4d9e-8a81-23892fbe03f8 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:01.958136 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec73c79-2d3f-4dca-8511-4ed40c55aa06 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:01.958417 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec73c79-2d3f-4dca-8511-4ed40c55aa06 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:01.958687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec73c79-2d3f-4dca-8511-4ed40c55aa06 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:01.959487 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec73c79-2d3f-4dca-8511-4ed40c55aa06 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:01.959818 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec73c79-2d3f-4dca-8511-4ed40c55aa06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:01.962514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec73c79-2d3f-4dca-8511-4ed40c55aa06 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:02.003517 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498d726-26b3-42ab-b18f-7c785389c3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:02.004259 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498d726-26b3-42ab-b18f-7c785389c3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:02.004545 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498d726-26b3-42ab-b18f-7c785389c3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:02.004864 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498d726-26b3-42ab-b18f-7c785389c3f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:02.018905 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498d726-26b3-42ab-b18f-7c785389c3f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:02.020568 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498d726-26b3-42ab-b18f-7c785389c3f2 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:02.038882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-7498d726-26b3-42ab-b18f-7c785389c3f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:02.038882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:02.038882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:02.038882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:02.038882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:02.038882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:02.038882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:02.038882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:02.038882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:02.038882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:02.038882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:02.038882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:02.038882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:02.038882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:02.038882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:02.039883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:02.039883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:02.039883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:02.039883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:02.039883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:02.039883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:02.039883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:02.039883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:02.039883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:02.039883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:02.039883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:02.039883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:02.039883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:02.039883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:02.039883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:02.039883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:02.039883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:02.040759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:02.040759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:02.040759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:02.040759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:02.045741 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 352/687] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:02.046760 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498d726-26b3-42ab-b18f-7c785389c3f2 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:02.082980 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:02.116578 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 336/688] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Jul 26 04:47:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:02.117545 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec73c79-2d3f-4dca-8511-4ed40c55aa06 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:02.164070 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498d726-26b3-42ab-b18f-7c785389c3f2 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:02.165645 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:02.208028 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039df8b-0c16-4fe5-8a44-9146b27cb3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:02.208645 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039df8b-0c16-4fe5-8a44-9146b27cb3c2 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:02.208926 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039df8b-0c16-4fe5-8a44-9146b27cb3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:02.209291 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039df8b-0c16-4fe5-8a44-9146b27cb3c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:02.210075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f039df8b-0c16-4fe5-8a44-9146b27cb3c2 None admin] RBAC: Adding query filter params (role.id=e5048f4e7c6b4494a15d628758090380, scope.project.id=d583e0be8ef74766bffd6c3c07af18f1, user.id=dbdb8997891241e9b2e829feb8023f8d) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:02.211063 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f039df8b-0c16-4fe5-8a44-9146b27cb3c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:02.212775 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f039df8b-0c16-4fe5-8a44-9146b27cb3c2 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:02.220184 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 353/689] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:47:02 2022] GET /identity/v3/role_assignments?user.id=dbdb8997891241e9b2e829feb8023f8d&scope.project.id=d583e0be8ef74766bffd6c3c07af18f1&role.id=e5048f4e7c6b4494a15d628758090380 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:02.220771 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039df8b-0c16-4fe5-8a44-9146b27cb3c2 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:02.268597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec73c79-2d3f-4dca-8511-4ed40c55aa06 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:02.313439 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd5de9c-6f6f-46ef-a15f-9ce6eebbea50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:02.314445 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd5de9c-6f6f-46ef-a15f-9ce6eebbea50 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:02.314445 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd5de9c-6f6f-46ef-a15f-9ce6eebbea50 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:02.314806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd5de9c-6f6f-46ef-a15f-9ce6eebbea50 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:02.315581 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd5de9c-6f6f-46ef-a15f-9ce6eebbea50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:02.315968 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd5de9c-6f6f-46ef-a15f-9ce6eebbea50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:02.317568 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd5de9c-6f6f-46ef-a15f-9ce6eebbea50 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:02.333346 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 337/690] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:02.337698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:02.347578 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd5de9c-6f6f-46ef-a15f-9ce6eebbea50 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:02.363434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039df8b-0c16-4fe5-8a44-9146b27cb3c2 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:02.407183 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c70cf-da42-4655-89ed-13dd54cf992e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:02.408222 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c70cf-da42-4655-89ed-13dd54cf992e None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:02.408730 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c70cf-da42-4655-89ed-13dd54cf992e None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:02.409198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c70cf-da42-4655-89ed-13dd54cf992e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:02.508790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c70cf-da42-4655-89ed-13dd54cf992e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:02.510547 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c70cf-da42-4655-89ed-13dd54cf992e None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:02.540236 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:02.557029 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd5de9c-6f6f-46ef-a15f-9ce6eebbea50 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:02.608224 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfdbc3b-2d4c-4e45-9ab0-d46e52860472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:02.609260 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdbc3b-2d4c-4e45-9ab0-d46e52860472 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:02.609744 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdbc3b-2d4c-4e45-9ab0-d46e52860472 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:02.610234 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdbc3b-2d4c-4e45-9ab0-d46e52860472 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:02.621367 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfdbc3b-2d4c-4e45-9ab0-d46e52860472 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:02.623442 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfdbc3b-2d4c-4e45-9ab0-d46e52860472 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:02.637765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:02.641117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-dcfdbc3b-2d4c-4e45-9ab0-d46e52860472 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 04:47:02.641117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:02.641117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:02.641117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:02.641117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:02.641117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:02.641117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:02.641117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:02.641117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:02.641117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:02.641117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:02.641117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:02.641117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:47:02.641117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:47:02.641117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:47:02.641117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:47:02.642175 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:02.642175 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:02.642175 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:47:02.642175 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:02.642175 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:47:02.642175 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:02.642175 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:02.642175 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:02.642175 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:02.642175 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:02.642175 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:02.642175 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:02.642175 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:02.642175 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:02.642175 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:02.642175 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:02.642175 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:02.642175 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:02.643085 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:02.643085 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:02.643085 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:47:02.643085 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:47:02.643085 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:47:02.643085 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:47:02.643085 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:47:02.643085 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:47:02.643085 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 04:47:02.643085 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:02.645345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 338/691] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Jul 26 04:47:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:02.649185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 354/692] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:47:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:47:02.653850 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfdbc3b-2d4c-4e45-9ab0-d46e52860472 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:02.655229 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c70cf-da42-4655-89ed-13dd54cf992e None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:02.769245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c70cf-da42-4655-89ed-13dd54cf992e None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:02.784850 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfdbc3b-2d4c-4e45-9ab0-d46e52860472 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:02.830927 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4acdd-a159-4fbc-a3de-0cabce0bd90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:02.831619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4acdd-a159-4fbc-a3de-0cabce0bd90e None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:02.831908 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4acdd-a159-4fbc-a3de-0cabce0bd90e None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:02.832178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4acdd-a159-4fbc-a3de-0cabce0bd90e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:02.847894 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d26b0-84c4-4d9c-8dc4-809d9dc7f8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:02.848711 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d26b0-84c4-4d9c-8dc4-809d9dc7f8dc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:02.848891 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d26b0-84c4-4d9c-8dc4-809d9dc7f8dc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:02.849161 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d26b0-84c4-4d9c-8dc4-809d9dc7f8dc None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:02.849982 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d26b0-84c4-4d9c-8dc4-809d9dc7f8dc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:02.850286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d26b0-84c4-4d9c-8dc4-809d9dc7f8dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:02.851783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d26b0-84c4-4d9c-8dc4-809d9dc7f8dc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:02.852263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4acdd-a159-4fbc-a3de-0cabce0bd90e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:02.853956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4acdd-a159-4fbc-a3de-0cabce0bd90e None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:02.869750 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-a3d4acdd-a159-4fbc-a3de-0cabce0bd90e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:02.869750 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:02.869750 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:02.869750 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:02.869750 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:02.869750 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:02.869750 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:02.869750 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:02.869750 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:02.869750 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:02.869750 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:02.869750 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:02.869750 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:02.869750 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:02.869750 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:02.870790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:02.870790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:02.870790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:02.870790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:02.870790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:02.870790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:02.870790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:02.870790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:02.870790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:02.870790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:02.870790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:02.870790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:02.870790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:02.870790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:02.870790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:02.870790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:02.870790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:02.871746 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:02.871746 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:02.871746 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:02.871746 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:02.873192 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 339/693] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Jul 26 04:47:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:02.874217 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 355/694] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:02.874533 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dbcb58-e163-4184-b23f-5667c0386d1c None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:02.874839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dbcb58-e163-4184-b23f-5667c0386d1c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:02.875972 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dbcb58-e163-4184-b23f-5667c0386d1c None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:02.881540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 340/695] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:02.882690 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dbcb58-e163-4184-b23f-5667c0386d1c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:02.882910 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4acdd-a159-4fbc-a3de-0cabce0bd90e None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:02.897429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:02.938241 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:03.003005 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4acdd-a159-4fbc-a3de-0cabce0bd90e None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:03.022251 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dbcb58-e163-4184-b23f-5667c0386d1c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:03.049689 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00089b15-f270-4a7d-adee-716b10b75831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:03.050967 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00089b15-f270-4a7d-adee-716b10b75831 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:03.051350 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00089b15-f270-4a7d-adee-716b10b75831 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:03.051749 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00089b15-f270-4a7d-adee-716b10b75831 None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:03.052843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00089b15-f270-4a7d-adee-716b10b75831 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:03.053246 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00089b15-f270-4a7d-adee-716b10b75831 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:03.055626 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00089b15-f270-4a7d-adee-716b10b75831 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:03.066412 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6a95b-57a5-4fd1-9218-0ffa97607baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:03.067193 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6a95b-57a5-4fd1-9218-0ffa97607baa None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:03.067597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6a95b-57a5-4fd1-9218-0ffa97607baa None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:03.068092 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6a95b-57a5-4fd1-9218-0ffa97607baa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:03.070495 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 356/696] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:03.072063 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdbb3f-81ee-4680-864a-696fee285385 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:03.072576 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdbb3f-81ee-4680-864a-696fee285385 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:03.073053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdbb3f-81ee-4680-864a-696fee285385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:03.092422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-abbdbb3f-81ee-4680-864a-696fee285385 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:03.093987 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-abbdbb3f-81ee-4680-864a-696fee285385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:03.160469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e6a95b-57a5-4fd1-9218-0ffa97607baa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:03.162599 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e6a95b-57a5-4fd1-9218-0ffa97607baa None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:03.169341 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 357/697] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:47:03.170503 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdbb3f-81ee-4680-864a-696fee285385 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:03.187143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:03.224220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:03.300602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdbb3f-81ee-4680-864a-696fee285385 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:03.307995 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 341/698] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:47:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:47:03.309057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6a95b-57a5-4fd1-9218-0ffa97607baa None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:03.346101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704e2ba-df60-4cc0-97ae-46cb9552df14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:03.347519 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704e2ba-df60-4cc0-97ae-46cb9552df14 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:03.347967 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704e2ba-df60-4cc0-97ae-46cb9552df14 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:03.348767 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704e2ba-df60-4cc0-97ae-46cb9552df14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:03.360606 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3704e2ba-df60-4cc0-97ae-46cb9552df14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:03.362403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3704e2ba-df60-4cc0-97ae-46cb9552df14 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:03.389991 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-3704e2ba-df60-4cc0-97ae-46cb9552df14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:03.389991 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:03.389991 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:03.389991 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:03.389991 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:03.389991 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:03.389991 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:03.389991 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:03.389991 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:03.389991 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:03.389991 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:03.389991 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:03.389991 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:47:03.389991 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:47:03.389991 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:47:03.391637 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:47:03.391637 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:03.391637 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:03.391637 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:03.391637 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:03.391637 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:03.391637 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:03.391637 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:03.391637 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:03.391637 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:03.391637 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:03.391637 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:03.391637 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:03.391637 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:03.391637 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:03.391637 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:03.391637 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:03.394233 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:47:03.394233 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:47:03.394233 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:47:03.394233 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:47:03.394233 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:47:03.394233 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:47:03.394233 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:03.394233 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:03.394233 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 358/699] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:47:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:03.394233 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704e2ba-df60-4cc0-97ae-46cb9552df14 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:03.477920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6a95b-57a5-4fd1-9218-0ffa97607baa None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:03.489469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:03.519807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e240f8-6fb7-41c2-be01-cc6c74a3991e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:03.520469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e240f8-6fb7-41c2-be01-cc6c74a3991e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:03.520750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e240f8-6fb7-41c2-be01-cc6c74a3991e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:03.521036 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e240f8-6fb7-41c2-be01-cc6c74a3991e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:03.521667 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e240f8-6fb7-41c2-be01-cc6c74a3991e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:03.523068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e240f8-6fb7-41c2-be01-cc6c74a3991e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:03.534702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e4e240f8-6fb7-41c2-be01-cc6c74a3991e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 04:47:03.534702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:03.534702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 04:47:03.534702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 04:47:03.534702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 04:47:03.534702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 04:47:03.534702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 04:47:03.534702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:03.534702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 04:47:03.534702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:03.534702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:47:03.534702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:03.534702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:03.534702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:03.534702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:03.534702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:03.535809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:03.535809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:03.535809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:03.535809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:03.535809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:03.535809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:03.535809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:03.535809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 04:47:03.535809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 04:47:03.535809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 04:47:03.535809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 04:47:03.535809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:03.535809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:03.535809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:03.535809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:03.535809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:03.536660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:03.536660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:03.536660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:03.536660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:03.536660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:03.536660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:03.536660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:03.536660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:03.536660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:03.536660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:03.536660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:03.536660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 04:47:03.536660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:03.536660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 04:47:03.536660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:03.536660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 342/700] 10.222.0.17 () {56 vars in 1129 bytes} [Tue Jul 26 04:47:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:03.537602 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e240f8-6fb7-41c2-be01-cc6c74a3991e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:03.545438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:03.552979 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704e2ba-df60-4cc0-97ae-46cb9552df14 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:03.598210 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa2d10-d890-4583-9bc9-ad5816e9ecec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:03.598878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa2d10-d890-4583-9bc9-ad5816e9ecec None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:03.599167 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa2d10-d890-4583-9bc9-ad5816e9ecec None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:03.599447 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa2d10-d890-4583-9bc9-ad5816e9ecec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:03.613218 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa2d10-d890-4583-9bc9-ad5816e9ecec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:03.614748 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa2d10-d890-4583-9bc9-ad5816e9ecec None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:03.634235 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-b1aa2d10-d890-4583-9bc9-ad5816e9ecec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:03.634235 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:03.634235 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:03.634235 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:03.634235 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:03.634235 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:03.634235 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:03.634235 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:03.634235 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:03.634235 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:03.634235 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:03.634235 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:03.634235 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:03.634235 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:03.634235 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:03.635234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:03.635234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:03.635234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:03.635234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:03.635234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:03.635234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:03.635234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:03.635234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:03.635234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:03.635234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:03.635234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:03.635234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:03.635234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:03.635234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:03.635234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:03.635234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:03.635234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:03.636427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:03.636427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:03.636427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:03.636427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:03.640371 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 359/701] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:03.641656 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2535a01-75e4-4650-bb9b-fdcb58a5c4ce None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:03.641944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2535a01-75e4-4650-bb9b-fdcb58a5c4ce None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:03.642244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2535a01-75e4-4650-bb9b-fdcb58a5c4ce None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:03.643852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 360/702] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:03.644564 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2535a01-75e4-4650-bb9b-fdcb58a5c4ce None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:03.669819 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e240f8-6fb7-41c2-be01-cc6c74a3991e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:03.708844 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6e0bc-a5e9-456b-8cc6-fc49740224af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:03.709502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6e0bc-a5e9-456b-8cc6-fc49740224af None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:03.709783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6e0bc-a5e9-456b-8cc6-fc49740224af None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:03.710055 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6e0bc-a5e9-456b-8cc6-fc49740224af None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:03.710933 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d6e0bc-a5e9-456b-8cc6-fc49740224af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:03.711234 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d6e0bc-a5e9-456b-8cc6-fc49740224af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:03.712862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d6e0bc-a5e9-456b-8cc6-fc49740224af None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:03.733920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 343/703] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:47:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:03.733920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6e0bc-a5e9-456b-8cc6-fc49740224af None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:03.770532 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:03.772366 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2535a01-75e4-4650-bb9b-fdcb58a5c4ce None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:03.813190 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a3c3c-b116-4cd3-8c18-f66160e21c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:03.813847 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a3c3c-b116-4cd3-8c18-f66160e21c52 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:03.814130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a3c3c-b116-4cd3-8c18-f66160e21c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:03.814405 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a3c3c-b116-4cd3-8c18-f66160e21c52 None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:03.815131 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a3c3c-b116-4cd3-8c18-f66160e21c52 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:03.815433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a3c3c-b116-4cd3-8c18-f66160e21c52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:03.816867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a3c3c-b116-4cd3-8c18-f66160e21c52 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:03.826976 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 361/704] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Jul 26 04:47:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:03.828408 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1ae7d-c6cc-43ed-b923-9da3a00346c7 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:03.828698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1ae7d-c6cc-43ed-b923-9da3a00346c7 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:03.828973 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1ae7d-c6cc-43ed-b923-9da3a00346c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:03.847593 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6e0bc-a5e9-456b-8cc6-fc49740224af None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:03.849153 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-7ed1ae7d-c6cc-43ed-b923-9da3a00346c7 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:03.850473 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-7ed1ae7d-c6cc-43ed-b923-9da3a00346c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:03.859776 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:03.890356 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e480b3-7413-446f-8d75-59d5ed8d5bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:03.891016 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e480b3-7413-446f-8d75-59d5ed8d5bcc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:03.891301 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e480b3-7413-446f-8d75-59d5ed8d5bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:03.891666 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e480b3-7413-446f-8d75-59d5ed8d5bcc None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:03.892415 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e480b3-7413-446f-8d75-59d5ed8d5bcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:03.892764 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e480b3-7413-446f-8d75-59d5ed8d5bcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:03.894200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e480b3-7413-446f-8d75-59d5ed8d5bcc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:03.915468 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 344/705] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:03.915468 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e480b3-7413-446f-8d75-59d5ed8d5bcc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:03.927600 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 362/706] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:47:03.928483 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1ae7d-c6cc-43ed-b923-9da3a00346c7 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:04.040502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:04.047099 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1ae7d-c6cc-43ed-b923-9da3a00346c7 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:04.051021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e480b3-7413-446f-8d75-59d5ed8d5bcc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:04.085418 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd90f3c-7ec2-400b-a3ec-91fd11e74a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:04.085922 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd90f3c-7ec2-400b-a3ec-91fd11e74a06 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:04.086203 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd90f3c-7ec2-400b-a3ec-91fd11e74a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:04.086555 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd90f3c-7ec2-400b-a3ec-91fd11e74a06 None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:04.087128 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd90f3c-7ec2-400b-a3ec-91fd11e74a06 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:04.087431 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd90f3c-7ec2-400b-a3ec-91fd11e74a06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:04.088676 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a753b9-f620-4875-87b4-390fb35e95f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:04.088931 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd90f3c-7ec2-400b-a3ec-91fd11e74a06 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:04.089342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a753b9-f620-4875-87b4-390fb35e95f5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:04.089622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a753b9-f620-4875-87b4-390fb35e95f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:04.089945 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a753b9-f620-4875-87b4-390fb35e95f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:04.090736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a753b9-f620-4875-87b4-390fb35e95f5 None admin] RBAC: Adding query filter params (role.id=e5048f4e7c6b4494a15d628758090380, scope.project.id=d583e0be8ef74766bffd6c3c07af18f1, user.id=d5218319aaa5424882714bffcbeee730) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:04.091046 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a753b9-f620-4875-87b4-390fb35e95f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:04.092874 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a753b9-f620-4875-87b4-390fb35e95f5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:04.108221 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 363/707] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Jul 26 04:47:03 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:04.108221 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd90f3c-7ec2-400b-a3ec-91fd11e74a06 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:04.111887 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 345/708] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:47:03 2022] GET /identity/v3/role_assignments?user.id=d5218319aaa5424882714bffcbeee730&scope.project.id=d583e0be8ef74766bffd6c3c07af18f1&role.id=e5048f4e7c6b4494a15d628758090380 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:04.112954 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a753b9-f620-4875-87b4-390fb35e95f5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:04.117528 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:04.273534 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd90f3c-7ec2-400b-a3ec-91fd11e74a06 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:04.291999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a753b9-f620-4875-87b4-390fb35e95f5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:04.355126 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff33c2-c09d-4945-b27e-c5cedfb1d445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:04.355760 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff33c2-c09d-4945-b27e-c5cedfb1d445 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:04.356043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff33c2-c09d-4945-b27e-c5cedfb1d445 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:04.356323 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff33c2-c09d-4945-b27e-c5cedfb1d445 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:04.365676 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff33c2-c09d-4945-b27e-c5cedfb1d445 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:04.366946 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19e997-b470-4ae0-9bd1-9b6f9f9b020f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:04.367301 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff33c2-c09d-4945-b27e-c5cedfb1d445 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:04.368115 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19e997-b470-4ae0-9bd1-9b6f9f9b020f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:04.368596 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19e997-b470-4ae0-9bd1-9b6f9f9b020f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:04.369056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19e997-b470-4ae0-9bd1-9b6f9f9b020f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:04.381676 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef19e997-b470-4ae0-9bd1-9b6f9f9b020f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:04.382826 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-30ff33c2-c09d-4945-b27e-c5cedfb1d445 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:04.382826 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:04.382826 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:04.382826 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:04.382826 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:04.382826 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:04.382826 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:04.382826 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:04.382826 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:04.382826 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:04.382826 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:04.382826 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:04.382826 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:47:04.382826 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:47:04.382826 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:47:04.383802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:47:04.383802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:04.383802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:04.383802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:04.383802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:04.383802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:04.383802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:04.383802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:04.383802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:04.383802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:04.383802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:04.383802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:04.383802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:04.383802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:04.383802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:04.383802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:04.383802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:04.384694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:47:04.384694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:47:04.384694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:47:04.384694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:47:04.384694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:47:04.384694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:47:04.384694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:04.384694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:04.384694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 364/709] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:47:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:04.386555 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef19e997-b470-4ae0-9bd1-9b6f9f9b020f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:04.391378 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff33c2-c09d-4945-b27e-c5cedfb1d445 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:04.404801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-ef19e997-b470-4ae0-9bd1-9b6f9f9b020f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:04.404801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:04.404801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:04.404801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:04.404801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:04.404801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:04.404801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:04.404801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:04.404801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:04.404801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:04.404801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:04.404801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:04.404801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:04.404801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:04.404801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:04.405757 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:04.405757 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:04.405757 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:04.405757 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:04.405757 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:04.405757 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:04.405757 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:04.405757 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:04.405757 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:04.405757 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:04.405757 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:04.405757 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:04.405757 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:04.405757 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:04.405757 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:04.405757 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:04.405757 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:04.406630 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:04.406630 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:04.406630 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:04.406630 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:04.408692 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 346/710] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:04.415998 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19e997-b470-4ae0-9bd1-9b6f9f9b020f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:04.429199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:04.472448 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:04.519130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff33c2-c09d-4945-b27e-c5cedfb1d445 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:04.545657 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19e997-b470-4ae0-9bd1-9b6f9f9b020f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:04.567117 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5136d5-787a-43ee-aac1-084f25b97a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:04.568327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5136d5-787a-43ee-aac1-084f25b97a62 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:04.568765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5136d5-787a-43ee-aac1-084f25b97a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:04.569546 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5136d5-787a-43ee-aac1-084f25b97a62 None admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:04.570935 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5136d5-787a-43ee-aac1-084f25b97a62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:04.571404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5136d5-787a-43ee-aac1-084f25b97a62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:04.573091 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5136d5-787a-43ee-aac1-084f25b97a62 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:04.596351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 365/711] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:47:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:04.604995 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5136d5-787a-43ee-aac1-084f25b97a62 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:04.607984 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22775c70-d4fc-4e05-88ac-b2d8bc119fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:04.608854 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22775c70-d4fc-4e05-88ac-b2d8bc119fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:04.609134 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22775c70-d4fc-4e05-88ac-b2d8bc119fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:04.610414 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22775c70-d4fc-4e05-88ac-b2d8bc119fd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:04.611199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22775c70-d4fc-4e05-88ac-b2d8bc119fd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:04.611503 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22775c70-d4fc-4e05-88ac-b2d8bc119fd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:04.613003 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22775c70-d4fc-4e05-88ac-b2d8bc119fd3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:04.629582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 347/712] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:04.636033 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22775c70-d4fc-4e05-88ac-b2d8bc119fd3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:04.744933 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:04.763395 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5136d5-787a-43ee-aac1-084f25b97a62 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:04.797871 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:04.807059 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22775c70-d4fc-4e05-88ac-b2d8bc119fd3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:04.808235 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242ff40-0f42-4eb6-9323-57213c399fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:04.808998 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242ff40-0f42-4eb6-9323-57213c399fdb None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:04.809375 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242ff40-0f42-4eb6-9323-57213c399fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:04.809744 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242ff40-0f42-4eb6-9323-57213c399fdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:04.810988 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f242ff40-0f42-4eb6-9323-57213c399fdb None admin] RBAC: Adding query filter params (role.id=72bf8bd23d2c4b76ad32b7b35d6db412, scope.project.id=d583e0be8ef74766bffd6c3c07af18f1, user.id=2faeed9002ee4154bc980c146cd06bcc) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:04.811398 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f242ff40-0f42-4eb6-9323-57213c399fdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:04.814036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f242ff40-0f42-4eb6-9323-57213c399fdb None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:04.831261 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 366/713] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:47:04 2022] GET /identity/v3/role_assignments?user.id=2faeed9002ee4154bc980c146cd06bcc&scope.project.id=d583e0be8ef74766bffd6c3c07af18f1&role.id=72bf8bd23d2c4b76ad32b7b35d6db412 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:04.833545 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eaca8f-a599-49ad-90a7-ba8be219deaf None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:04.834619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eaca8f-a599-49ad-90a7-ba8be219deaf None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:04.834619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eaca8f-a599-49ad-90a7-ba8be219deaf None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:04.842135 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 367/714] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:04.846800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d6360-e8f9-493f-8420-5394e273b164 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:04.847104 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d6360-e8f9-493f-8420-5394e273b164 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:04.847497 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d6360-e8f9-493f-8420-5394e273b164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:04.903639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-f63d6360-e8f9-493f-8420-5394e273b164 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:04.905746 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-f63d6360-e8f9-493f-8420-5394e273b164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:04.921047 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d3778-0c81-4585-8809-02ca4eb5a481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:04.922116 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d3778-0c81-4585-8809-02ca4eb5a481 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:04.922619 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d3778-0c81-4585-8809-02ca4eb5a481 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:04.923122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d3778-0c81-4585-8809-02ca4eb5a481 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:04.929096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d3778-0c81-4585-8809-02ca4eb5a481 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:04.931977 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d3778-0c81-4585-8809-02ca4eb5a481 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:05.001188 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 348/715] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jul 26 04:47:04 2022] POST /identity/v3/users => generated 254 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 04:47:05.073463 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 368/716] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 04:47:05.131655 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d3778-0c81-4585-8809-02ca4eb5a481 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:05.336846 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d3778-0c81-4585-8809-02ca4eb5a481 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:05.383900 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:05.405505 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce2ffa-83d2-4b29-98cd-67cb041dae33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:05.406192 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce2ffa-83d2-4b29-98cd-67cb041dae33 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:05.406478 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce2ffa-83d2-4b29-98cd-67cb041dae33 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:05.406750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce2ffa-83d2-4b29-98cd-67cb041dae33 None admin] PATH_INFO: `/v3/services/network` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:05.407405 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce2ffa-83d2-4b29-98cd-67cb041dae33 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:05.408842 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce2ffa-83d2-4b29-98cd-67cb041dae33 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:05.421371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d5ce2ffa-83d2-4b29-98cd-67cb041dae33 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 04:47:05.421371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:05.421371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 04:47:05.421371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 04:47:05.421371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 04:47:05.421371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 04:47:05.421371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 04:47:05.421371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:05.421371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 04:47:05.421371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:05.421371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:47:05.421371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:05.421371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:05.421371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:05.421371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:05.421371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:05.421371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:05.422437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:05.422437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:05.422437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:05.422437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:05.422437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:05.422437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:05.422437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 04:47:05.422437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 04:47:05.422437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 04:47:05.422437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 04:47:05.422437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:05.422437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:05.422437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:05.422437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:05.422437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:05.422437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:05.423286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:05.423286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:05.423286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:05.423286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:05.423286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:05.423286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:05.423286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:05.423286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:05.423286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:05.423286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:05.423286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 04:47:05.423286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:05.423286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 04:47:05.423286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:05.423286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 349/717] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:47:05 2022] GET /identity/v3/services/network => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:05.429443 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d6360-e8f9-493f-8420-5394e273b164 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:05.594307 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d6360-e8f9-493f-8420-5394e273b164 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:05.630647 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:05.654737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a067316-8bf7-4a19-9b7b-88f64b23bb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:05.655389 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a067316-8bf7-4a19-9b7b-88f64b23bb43 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:05.655699 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a067316-8bf7-4a19-9b7b-88f64b23bb43 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:05.655979 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a067316-8bf7-4a19-9b7b-88f64b23bb43 None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:05.656682 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a067316-8bf7-4a19-9b7b-88f64b23bb43 None admin] RBAC: Adding query filter params (name=network) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:05.656988 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a067316-8bf7-4a19-9b7b-88f64b23bb43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:05.658390 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a067316-8bf7-4a19-9b7b-88f64b23bb43 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:05.667698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 369/718] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Jul 26 04:47:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:05.670082 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce2ffa-83d2-4b29-98cd-67cb041dae33 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:05.825874 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce2ffa-83d2-4b29-98cd-67cb041dae33 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:05.867196 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd449625-c684-40e7-aa2a-00b4d3c6c7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:05.867805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd449625-c684-40e7-aa2a-00b4d3c6c7fb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:05.868089 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd449625-c684-40e7-aa2a-00b4d3c6c7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:05.868364 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd449625-c684-40e7-aa2a-00b4d3c6c7fb None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:05.869072 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd449625-c684-40e7-aa2a-00b4d3c6c7fb None admin] RBAC: Adding query filter params (type=network) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:05.869378 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd449625-c684-40e7-aa2a-00b4d3c6c7fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:05.870881 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd449625-c684-40e7-aa2a-00b4d3c6c7fb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:05.878018 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 350/719] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Jul 26 04:47:05 2022] GET /identity/v3/services?type=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:06.553852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2491168-4686-4950-82d8-12c110d59525 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:06.554907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2491168-4686-4950-82d8-12c110d59525 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:06.555397 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2491168-4686-4950-82d8-12c110d59525 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:06.557398 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 370/720] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:06.563474 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c97fa-fa58-49a3-896d-8cb009976d8b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:06.564127 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c97fa-fa58-49a3-896d-8cb009976d8b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:06.564635 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c97fa-fa58-49a3-896d-8cb009976d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:06.592788 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ee9c97fa-fa58-49a3-896d-8cb009976d8b None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:06.594428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ee9c97fa-fa58-49a3-896d-8cb009976d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:06.672766 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:06.686525 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b101bc8-fde2-4489-8dfd-0cc58809ca07 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:06.686807 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b101bc8-fde2-4489-8dfd-0cc58809ca07 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:06.687086 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b101bc8-fde2-4489-8dfd-0cc58809ca07 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:06.690005 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 371/721] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:06.694665 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a531f38-ad29-4813-bfbc-7acbd1333fdc None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:06.695025 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a531f38-ad29-4813-bfbc-7acbd1333fdc None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:06.695204 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a531f38-ad29-4813-bfbc-7acbd1333fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:06.731520 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 351/722] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:47:06.737655 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-4a531f38-ad29-4813-bfbc-7acbd1333fdc None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:06.739763 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-4a531f38-ad29-4813-bfbc-7acbd1333fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:06.794065 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c97fa-fa58-49a3-896d-8cb009976d8b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:06.873526 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 372/723] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 04:47:06.897322 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a531f38-ad29-4813-bfbc-7acbd1333fdc None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:06.985546 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c97fa-fa58-49a3-896d-8cb009976d8b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:07.024364 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5020212-23d7-45b6-9f3f-660a8e8c68d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:07.025006 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5020212-23d7-45b6-9f3f-660a8e8c68d7 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:07.025283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5020212-23d7-45b6-9f3f-660a8e8c68d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:07.025555 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5020212-23d7-45b6-9f3f-660a8e8c68d7 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:07.026322 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5020212-23d7-45b6-9f3f-660a8e8c68d7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:07.027783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5020212-23d7-45b6-9f3f-660a8e8c68d7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:07.050928 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 352/724] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jul 26 04:47:06 2022] POST /identity/v3/services => generated 251 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 04:47:07.058247 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a531f38-ad29-4813-bfbc-7acbd1333fdc None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:07.081212 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:07.129513 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb5ebc3-6298-4996-930c-c5a809e3847c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:07.130410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5ebc3-6298-4996-930c-c5a809e3847c None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:07.130815 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5ebc3-6298-4996-930c-c5a809e3847c None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:07.131207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5ebc3-6298-4996-930c-c5a809e3847c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:07.151674 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb5ebc3-6298-4996-930c-c5a809e3847c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:47:07.156873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-2eb5ebc3-6298-4996-930c-c5a809e3847c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:47:07.156873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:07.156873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:07.156873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:07.156873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:07.156873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:07.156873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:07.156873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:07.156873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:07.156873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:07.156873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:07.156873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:07.156873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:47:07.156873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:47:07.156873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:47:07.156873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:47:07.157901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:47:07.157901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:47:07.157901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:47:07.157901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:47:07.157901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:07.157901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:07.157901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:07.157901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:07.157901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:07.157901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:07.157901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:07.157901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:07.157901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:07.157901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:07.157901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:07.157901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:07.158769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:07.158769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:07.158769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:07.158769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:07.158769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:47:07.158769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:47:07.158769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:47:07.158769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:47:07.158769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:47:07.158769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:47:07.158769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:47:07.158769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:07.158769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 373/725] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:47:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:07.164605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5020212-23d7-45b6-9f3f-660a8e8c68d7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:07.228566 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:07.315942 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5020212-23d7-45b6-9f3f-660a8e8c68d7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:07.359473 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c387d1c-0c19-447e-b229-6a590bd5b50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:07.360158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c387d1c-0c19-447e-b229-6a590bd5b50b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:07.360441 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c387d1c-0c19-447e-b229-6a590bd5b50b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:07.360716 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c387d1c-0c19-447e-b229-6a590bd5b50b None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:07.361460 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c387d1c-0c19-447e-b229-6a590bd5b50b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:07.361764 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c387d1c-0c19-447e-b229-6a590bd5b50b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:07.363149 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c387d1c-0c19-447e-b229-6a590bd5b50b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:07.373841 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 353/726] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:47:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:07.379903 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb5ebc3-6298-4996-930c-c5a809e3847c None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:07.397321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490dbaa-9ce7-493c-8fdb-bae58c147cbe None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:07.397609 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490dbaa-9ce7-493c-8fdb-bae58c147cbe None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:07.397898 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490dbaa-9ce7-493c-8fdb-bae58c147cbe None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:07.399461 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 354/727] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:47:07.405259 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfdeb7f-9c34-44ae-87e9-836c0ea78fd8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:07.405559 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfdeb7f-9c34-44ae-87e9-836c0ea78fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:07.405815 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfdeb7f-9c34-44ae-87e9-836c0ea78fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:07.434534 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-bdfdeb7f-9c34-44ae-87e9-836c0ea78fd8 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:07.435949 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bdfdeb7f-9c34-44ae-87e9-836c0ea78fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:07.547922 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 355/728] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:07 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 04:47:07.576926 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb5ebc3-6298-4996-930c-c5a809e3847c None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:07.620712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfdeb7f-9c34-44ae-87e9-836c0ea78fd8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:07.632514 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7747d76-2fda-4d34-ab57-ea0238c4b8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:07.633190 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7747d76-2fda-4d34-ab57-ea0238c4b8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:07.633472 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7747d76-2fda-4d34-ab57-ea0238c4b8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:07.633744 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7747d76-2fda-4d34-ab57-ea0238c4b8f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:07.653420 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7747d76-2fda-4d34-ab57-ea0238c4b8f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:07.656736 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7747d76-2fda-4d34-ab57-ea0238c4b8f0 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:07.691445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-e7747d76-2fda-4d34-ab57-ea0238c4b8f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:07.691445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:07.691445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:07.691445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:07.691445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:07.691445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:07.691445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:07.691445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:07.691445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:07.691445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:07.691445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:07.691445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:07.691445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:07.691445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:07.691445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:07.692427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:07.692427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:07.692427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:07.692427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:07.692427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:07.692427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:07.692427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:07.692427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:07.692427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:07.692427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:07.692427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:07.692427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:07.692427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:07.692427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:07.692427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:07.692427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:07.692427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:07.693310 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:07.693310 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:07.693310 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:07.693310 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:07.697584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:07.699593 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 374/729] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:07.705513 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7747d76-2fda-4d34-ab57-ea0238c4b8f0 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:07.738331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:07.821553 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7747d76-2fda-4d34-ab57-ea0238c4b8f0 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:07.856781 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfdeb7f-9c34-44ae-87e9-836c0ea78fd8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:07.871363 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15011c89-deb6-4d34-8c52-9ece35bb8f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:07.872465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15011c89-deb6-4d34-8c52-9ece35bb8f6c None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:07.872975 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15011c89-deb6-4d34-8c52-9ece35bb8f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:07.873454 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15011c89-deb6-4d34-8c52-9ece35bb8f6c None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:07.874374 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15011c89-deb6-4d34-8c52-9ece35bb8f6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:07.874891 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15011c89-deb6-4d34-8c52-9ece35bb8f6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:07.876666 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15011c89-deb6-4d34-8c52-9ece35bb8f6c None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:07.898694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 375/730] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:07.905842 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15011c89-deb6-4d34-8c52-9ece35bb8f6c None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:07.919264 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a72759-605d-48cb-a83c-7b912cd9dd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:07.920429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a72759-605d-48cb-a83c-7b912cd9dd06 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:07.920919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a72759-605d-48cb-a83c-7b912cd9dd06 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:07.921419 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a72759-605d-48cb-a83c-7b912cd9dd06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:07.944904 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a72759-605d-48cb-a83c-7b912cd9dd06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:47:07.952966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-09a72759-605d-48cb-a83c-7b912cd9dd06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:47:07.952966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:07.952966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:07.952966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:07.952966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:07.952966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:07.952966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:07.952966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:07.952966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:07.952966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:07.952966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:07.952966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:07.952966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:47:07.952966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:47:07.952966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:47:07.952966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:47:07.953976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:47:07.953976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:47:07.953976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:47:07.953976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:47:07.953976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:07.953976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:07.953976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:07.953976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:07.953976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:07.953976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:07.953976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:07.953976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:07.953976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:07.953976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:07.953976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:07.953976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:07.954803 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:07.954803 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:07.954803 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:07.954803 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:07.954803 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:47:07.954803 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:47:07.954803 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:47:07.954803 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:47:07.954803 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:47:07.954803 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:47:07.954803 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:47:07.954803 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:07.954803 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 356/731] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:47:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:47:07.972665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a72759-605d-48cb-a83c-7b912cd9dd06 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:08.038945 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:08.064735 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15011c89-deb6-4d34-8c52-9ece35bb8f6c None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:08.102406 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e7220-fe4f-40e5-8209-28ce9d780498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:08.102639 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:08.104133 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e7220-fe4f-40e5-8209-28ce9d780498 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:08.104763 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e7220-fe4f-40e5-8209-28ce9d780498 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:08.105274 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e7220-fe4f-40e5-8209-28ce9d780498 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:08.112328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a72759-605d-48cb-a83c-7b912cd9dd06 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:08.114156 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e7220-fe4f-40e5-8209-28ce9d780498 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:08.115891 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e7220-fe4f-40e5-8209-28ce9d780498 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:08.128965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-2c6e7220-fe4f-40e5-8209-28ce9d780498 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 04:47:08.128965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:08.128965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:08.128965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:08.128965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:08.128965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:08.128965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:08.128965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:08.128965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:08.128965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:08.128965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:08.128965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:08.128965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:47:08.128965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:47:08.128965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:47:08.128965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:47:08.129977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:08.129977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:08.129977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:47:08.129977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:08.129977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:47:08.129977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:08.129977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:08.129977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:08.129977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:08.129977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:08.129977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:08.129977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:08.129977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:08.129977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:08.129977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:08.129977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:08.129977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:08.129977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:08.130926 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:08.130926 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:08.130926 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:47:08.130926 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:47:08.130926 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:47:08.130926 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:47:08.130926 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:47:08.130926 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:47:08.130926 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 04:47:08.130926 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:08.133593 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 376/732] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:47:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:47:08.139587 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e7220-fe4f-40e5-8209-28ce9d780498 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:08.176264 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12bb8f-af6e-422a-bf16-5e764d16c1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:08.177397 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12bb8f-af6e-422a-bf16-5e764d16c1df None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:08.177885 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12bb8f-af6e-422a-bf16-5e764d16c1df None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:08.178360 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12bb8f-af6e-422a-bf16-5e764d16c1df None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:08.179498 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a12bb8f-af6e-422a-bf16-5e764d16c1df None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:08.181046 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a12bb8f-af6e-422a-bf16-5e764d16c1df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:08.183492 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a12bb8f-af6e-422a-bf16-5e764d16c1df None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:08.200402 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 357/733] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:47:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:08.206446 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87365220-d434-4e34-b45d-0c24f6ac952e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:08.206446 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87365220-d434-4e34-b45d-0c24f6ac952e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:08.206446 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87365220-d434-4e34-b45d-0c24f6ac952e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:08.207619 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 358/734] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:08.208707 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87365220-d434-4e34-b45d-0c24f6ac952e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:08.300737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e7220-fe4f-40e5-8209-28ce9d780498 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:08.332778 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87365220-d434-4e34-b45d-0c24f6ac952e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:08.355931 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42938f77-b355-467e-af9b-48b2022566fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:08.357578 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42938f77-b355-467e-af9b-48b2022566fa None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:08.358118 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42938f77-b355-467e-af9b-48b2022566fa None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:08.358881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42938f77-b355-467e-af9b-48b2022566fa None admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:08.360198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42938f77-b355-467e-af9b-48b2022566fa None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:08.360736 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42938f77-b355-467e-af9b-48b2022566fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:08.363110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42938f77-b355-467e-af9b-48b2022566fa None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:08.372872 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b87731-0021-4ec1-87b9-4394d9e91931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:08.373535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b87731-0021-4ec1-87b9-4394d9e91931 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:08.373818 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b87731-0021-4ec1-87b9-4394d9e91931 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:08.374094 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b87731-0021-4ec1-87b9-4394d9e91931 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:08.383187 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:08.388995 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b87731-0021-4ec1-87b9-4394d9e91931 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:08.391636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b87731-0021-4ec1-87b9-4394d9e91931 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:08.395370 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 377/735] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Jul 26 04:47:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:08.397471 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f13e6e-c645-4e06-8aac-6154cf78f8ae None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:08.397833 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f13e6e-c645-4e06-8aac-6154cf78f8ae None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:08.398183 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f13e6e-c645-4e06-8aac-6154cf78f8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:08.410629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b8b87731-0021-4ec1-87b9-4394d9e91931 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:08.410629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:08.410629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:08.410629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:08.410629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:08.410629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:08.410629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:08.410629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:08.410629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:08.410629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:08.410629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:08.410629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:08.410629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:08.410629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:08.410629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:08.411715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:08.411715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:08.411715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:08.411715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:08.411715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:08.411715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:08.411715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:08.411715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:08.411715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:08.411715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:08.411715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:08.411715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:08.411715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:08.411715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:08.411715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:08.411715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:08.411715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:08.412607 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:08.412607 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:08.412607 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:08.412607 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:08.419256 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 359/736] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:08.419414 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-66f13e6e-c645-4e06-8aac-6154cf78f8ae None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:08.419950 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-66f13e6e-c645-4e06-8aac-6154cf78f8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:08.420745 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b87731-0021-4ec1-87b9-4394d9e91931 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:08.434384 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:08.515588 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 378/737] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:08 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:47:08.515588 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f13e6e-c645-4e06-8aac-6154cf78f8ae None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:08.567889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b87731-0021-4ec1-87b9-4394d9e91931 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:08.616405 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c9c7f-3d42-41f9-853b-8ee6421f7a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:08.617386 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c9c7f-3d42-41f9-853b-8ee6421f7a01 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:08.617764 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c9c7f-3d42-41f9-853b-8ee6421f7a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:08.617968 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c9c7f-3d42-41f9-853b-8ee6421f7a01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:08.645162 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f13e6e-c645-4e06-8aac-6154cf78f8ae None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:08.683267 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:08.694946 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7cd24-3296-4909-b0c2-c8387e7f12f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:08.695623 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7cd24-3296-4909-b0c2-c8387e7f12f5 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:08.695913 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7cd24-3296-4909-b0c2-c8387e7f12f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:08.696186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7cd24-3296-4909-b0c2-c8387e7f12f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:08.700106 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e7cd24-3296-4909-b0c2-c8387e7f12f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:08.700106 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e7cd24-3296-4909-b0c2-c8387e7f12f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:08.700106 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e7cd24-3296-4909-b0c2-c8387e7f12f5 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:08.719771 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 379/738] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:08.719771 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7cd24-3296-4909-b0c2-c8387e7f12f5 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:08.722700 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c9c7f-3d42-41f9-853b-8ee6421f7a01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:08.724605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c9c7f-3d42-41f9-853b-8ee6421f7a01 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:08.743502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:08.823127 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7cd24-3296-4909-b0c2-c8387e7f12f5 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:08.854863 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 360/739] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:47:08 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 04:47:08.856214 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c10f19-4c4f-4291-9e36-39faaa679c91 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:08.856506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c10f19-4c4f-4291-9e36-39faaa679c91 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:08.856785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c10f19-4c4f-4291-9e36-39faaa679c91 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:08.858220 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 361/740] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:08.862817 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c10f19-4c4f-4291-9e36-39faaa679c91 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:08.864874 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d7b92-854e-4caf-869a-bf27aa5d12de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:08.865507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d7b92-854e-4caf-869a-bf27aa5d12de None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:08.865822 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d7b92-854e-4caf-869a-bf27aa5d12de None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:08.866093 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d7b92-854e-4caf-869a-bf27aa5d12de None admin] PATH_INFO: `/v3/roles/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:08.879085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d7b92-854e-4caf-869a-bf27aa5d12de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:47:08.884608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-cb0d7b92-854e-4caf-869a-bf27aa5d12de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:47:08.884608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:08.884608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:08.884608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:08.884608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:08.884608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:08.884608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:08.884608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:08.884608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:08.884608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:08.884608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:08.884608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:08.884608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:47:08.884608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:47:08.884608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:47:08.884608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:47:08.885572 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:47:08.885572 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:47:08.885572 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:47:08.885572 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:47:08.885572 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:08.885572 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:08.885572 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:08.885572 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:08.885572 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:08.885572 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:08.885572 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:08.885572 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:08.885572 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:08.885572 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:08.885572 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:08.885572 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:08.886413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:08.886413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:08.886413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:08.886413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:08.886413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:47:08.886413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:47:08.886413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:47:08.886413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:47:08.886413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:47:08.886413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:47:08.886413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:47:08.886413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:08.890806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 380/741] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jul 26 04:47:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:08.891759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d7b92-854e-4caf-869a-bf27aa5d12de None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:08.952008 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:08.983717 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c10f19-4c4f-4291-9e36-39faaa679c91 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:08.995818 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:09.010555 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d7b92-854e-4caf-869a-bf27aa5d12de None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:09.023904 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acde62d2-3e90-41ba-a2ad-76e8cff34f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:09.024482 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde62d2-3e90-41ba-a2ad-76e8cff34f4b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:09.024766 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde62d2-3e90-41ba-a2ad-76e8cff34f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:09.025033 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde62d2-3e90-41ba-a2ad-76e8cff34f4b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:09.032834 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acde62d2-3e90-41ba-a2ad-76e8cff34f4b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:09.034321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acde62d2-3e90-41ba-a2ad-76e8cff34f4b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:09.046963 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-acde62d2-3e90-41ba-a2ad-76e8cff34f4b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 04:47:09.046963 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:09.046963 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:09.046963 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:09.046963 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:09.046963 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:09.046963 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:09.046963 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:09.046963 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:09.046963 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:09.046963 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:09.046963 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:09.046963 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:47:09.046963 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:47:09.046963 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:47:09.046963 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:47:09.048164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:09.048164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:09.048164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:47:09.048164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:09.048164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:47:09.048164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:09.048164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:09.048164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:09.048164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:09.048164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:09.048164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:09.048164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:09.048164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:09.048164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:09.048164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:09.048164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:09.048164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:09.048164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:09.049083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:09.049083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:09.049083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:47:09.049083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:47:09.049083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:47:09.049083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:47:09.049083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:47:09.049083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:47:09.049083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 04:47:09.049083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:09.049083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 362/742] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Jul 26 04:47:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:09.049780 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa713d4e-c0f2-4e83-8ff7-d321b78f7759 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:09.049985 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa713d4e-c0f2-4e83-8ff7-d321b78f7759 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:09.050262 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa713d4e-c0f2-4e83-8ff7-d321b78f7759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:09.070226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fa713d4e-c0f2-4e83-8ff7-d321b78f7759 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:09.071429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fa713d4e-c0f2-4e83-8ff7-d321b78f7759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:09.078923 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f668b-bdc2-46c1-a6ec-3830e2f70773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:09.080047 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f668b-bdc2-46c1-a6ec-3830e2f70773 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:09.080543 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f668b-bdc2-46c1-a6ec-3830e2f70773 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:09.081040 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f668b-bdc2-46c1-a6ec-3830e2f70773 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:09.093548 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856f668b-bdc2-46c1-a6ec-3830e2f70773 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:09.095124 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856f668b-bdc2-46c1-a6ec-3830e2f70773 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:09.111036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-856f668b-bdc2-46c1-a6ec-3830e2f70773 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:09.111036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:09.111036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:09.111036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:09.111036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:09.111036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:09.111036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:09.111036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:09.111036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:09.111036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:09.111036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:09.111036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:09.111036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:09.111036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:09.111036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:09.112084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:09.112084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:09.112084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:09.112084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:09.112084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:09.112084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:09.112084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:09.112084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:09.112084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:09.112084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:09.112084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:09.112084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:09.112084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:09.112084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:09.112084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:09.112084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:09.112084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:09.113551 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:09.113551 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:09.113551 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:09.113551 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:09.113551 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 381/743] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:09.113551 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f668b-bdc2-46c1-a6ec-3830e2f70773 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:09.187063 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 363/744] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:09 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 04:47:09.188521 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa713d4e-c0f2-4e83-8ff7-d321b78f7759 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:09.236626 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f668b-bdc2-46c1-a6ec-3830e2f70773 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:09.239538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:09.303051 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af6dfd-c831-46c6-aecb-e67f2e408024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:09.304161 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af6dfd-c831-46c6-aecb-e67f2e408024 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:09.304669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af6dfd-c831-46c6-aecb-e67f2e408024 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:09.305146 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af6dfd-c831-46c6-aecb-e67f2e408024 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:09.305587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:09.306526 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0af6dfd-c831-46c6-aecb-e67f2e408024 None admin] RBAC: Adding query filter params (name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:09.306846 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0af6dfd-c831-46c6-aecb-e67f2e408024 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:09.309595 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0af6dfd-c831-46c6-aecb-e67f2e408024 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:09.310417 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa713d4e-c0f2-4e83-8ff7-d321b78f7759 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:09.329640 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 382/745] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jul 26 04:47:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:09.329640 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af6dfd-c831-46c6-aecb-e67f2e408024 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:09.358435 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4f3d4-22be-46c2-b544-c08ca71a709f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:09.359076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4f3d4-22be-46c2-b544-c08ca71a709f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:09.359367 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4f3d4-22be-46c2-b544-c08ca71a709f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:09.359689 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4f3d4-22be-46c2-b544-c08ca71a709f None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:09.360501 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea4f3d4-22be-46c2-b544-c08ca71a709f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:09.360811 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea4f3d4-22be-46c2-b544-c08ca71a709f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:09.362292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea4f3d4-22be-46c2-b544-c08ca71a709f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:09.380777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 364/746] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Jul 26 04:47:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:09.381607 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4f3d4-22be-46c2-b544-c08ca71a709f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:09.463155 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af6dfd-c831-46c6-aecb-e67f2e408024 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:09.492760 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4f3d4-22be-46c2-b544-c08ca71a709f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:09.499729 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0c933-5777-47b8-bab8-a93d813d325e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:09.500360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0c933-5777-47b8-bab8-a93d813d325e None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:09.500644 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0c933-5777-47b8-bab8-a93d813d325e None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:09.500916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0c933-5777-47b8-bab8-a93d813d325e None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:09.502718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e0c933-5777-47b8-bab8-a93d813d325e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:09.502718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e0c933-5777-47b8-bab8-a93d813d325e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:09.503303 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e0c933-5777-47b8-bab8-a93d813d325e None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:09.515197 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 383/747] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:09.516079 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0c933-5777-47b8-bab8-a93d813d325e None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:09.530709 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea51d2f-2f36-46f4-b4fc-a03a0dd989a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:09.531538 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea51d2f-2f36-46f4-b4fc-a03a0dd989a2 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:09.531948 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea51d2f-2f36-46f4-b4fc-a03a0dd989a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:09.532412 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea51d2f-2f36-46f4-b4fc-a03a0dd989a2 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:09.533225 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea51d2f-2f36-46f4-b4fc-a03a0dd989a2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:09.534789 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea51d2f-2f36-46f4-b4fc-a03a0dd989a2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:09.542978 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:09.562906 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 365/748] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jul 26 04:47:09 2022] POST /identity/v3/services => generated 240 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:47:09.562906 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea51d2f-2f36-46f4-b4fc-a03a0dd989a2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:09.593808 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:09.641066 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0c933-5777-47b8-bab8-a93d813d325e None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:09.685713 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e547f6e6-a812-44c9-9a63-11318a6a4bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:09.686392 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547f6e6-a812-44c9-9a63-11318a6a4bdc None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:09.686676 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547f6e6-a812-44c9-9a63-11318a6a4bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:09.686924 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547f6e6-a812-44c9-9a63-11318a6a4bdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:09.699828 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e547f6e6-a812-44c9-9a63-11318a6a4bdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:09.701457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e547f6e6-a812-44c9-9a63-11318a6a4bdc None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:09.718247 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-e547f6e6-a812-44c9-9a63-11318a6a4bdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:09.718247 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:09.718247 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:09.718247 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:09.718247 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:09.718247 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:09.718247 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:09.718247 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:09.718247 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:09.718247 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:09.718247 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:09.718247 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:09.718247 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:09.718247 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:09.718247 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:09.719192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:09.719192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:09.719192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:09.719192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:09.719192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:09.719192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:09.719192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:09.719192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:09.719192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:09.719192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:09.719192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:09.719192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:09.719192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:09.719192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:09.719192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:09.719192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:09.719192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:09.720147 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:09.720147 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:09.720147 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:09.720147 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:09.720147 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 384/749] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:09.720956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe5842-323d-4912-bec1-a7a368332403 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:09.720956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe5842-323d-4912-bec1-a7a368332403 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:09.721258 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe5842-323d-4912-bec1-a7a368332403 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:09.726715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 385/750] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:09.726715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe5842-323d-4912-bec1-a7a368332403 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:09.727572 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea51d2f-2f36-46f4-b4fc-a03a0dd989a2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:09.777941 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b28dee-a37f-414b-a9fa-082b2c512daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:09.778920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b28dee-a37f-414b-a9fa-082b2c512daa None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:09.779237 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b28dee-a37f-414b-a9fa-082b2c512daa None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:09.779488 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b28dee-a37f-414b-a9fa-082b2c512daa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:09.790798 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b28dee-a37f-414b-a9fa-082b2c512daa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:09.792525 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b28dee-a37f-414b-a9fa-082b2c512daa None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:09.807318 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-70b28dee-a37f-414b-a9fa-082b2c512daa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:09.807318 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:09.807318 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:09.807318 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:09.807318 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:09.807318 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:09.807318 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:09.807318 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:09.807318 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:09.807318 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:09.807318 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:09.807318 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:09.807318 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:09.807318 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:09.807318 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:09.808355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:09.808355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:09.808355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:09.808355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:09.808355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:09.808355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:09.808355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:09.808355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:09.808355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:09.808355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:09.808355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:09.808355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:09.808355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:09.808355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:09.808355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:09.808355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:09.808355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:09.809233 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:09.809233 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:09.809233 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:09.809233 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:09.809233 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 366/751] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:09.809502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b28dee-a37f-414b-a9fa-082b2c512daa None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:09.810225 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:09.847639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe5842-323d-4912-bec1-a7a368332403 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:09.889116 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93829d56-ff84-4aca-804e-5096aa0c2c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:09.889806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93829d56-ff84-4aca-804e-5096aa0c2c2b None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:09.889806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93829d56-ff84-4aca-804e-5096aa0c2c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:09.889806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93829d56-ff84-4aca-804e-5096aa0c2c2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:09.892500 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:09.901980 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93829d56-ff84-4aca-804e-5096aa0c2c2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:09.904694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93829d56-ff84-4aca-804e-5096aa0c2c2b None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:09.923902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-93829d56-ff84-4aca-804e-5096aa0c2c2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:09.923902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:09.923902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:09.923902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:09.923902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:09.923902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:09.923902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:09.923902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:09.923902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:09.923902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:09.923902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:09.923902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:09.923902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:47:09.923902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:47:09.923902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:47:09.924912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:47:09.924912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:09.924912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:09.924912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:09.924912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:09.924912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:09.924912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:09.924912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:09.924912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:09.924912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:09.924912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:09.924912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:09.924912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:09.924912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:09.924912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:09.924912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:09.924912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:09.925779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:47:09.925779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:47:09.925779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:47:09.925779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:47:09.925779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:47:09.925779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:47:09.925779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:09.925779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:09.925779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 386/752] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:47:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:09.927847 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24b900-9a2e-46e5-8800-6834ad397106 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:09.928337 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24b900-9a2e-46e5-8800-6834ad397106 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:09.928819 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24b900-9a2e-46e5-8800-6834ad397106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:09.957103 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-ab24b900-9a2e-46e5-8800-6834ad397106 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:09.958915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-ab24b900-9a2e-46e5-8800-6834ad397106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:09.964103 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b28dee-a37f-414b-a9fa-082b2c512daa None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:10.018020 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c65da4-9045-4bcb-b803-e202caa3b8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:10.018908 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c65da4-9045-4bcb-b803-e202caa3b8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:10.019360 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c65da4-9045-4bcb-b803-e202caa3b8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:10.019830 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c65da4-9045-4bcb-b803-e202caa3b8c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:10.020989 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c65da4-9045-4bcb-b803-e202caa3b8c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:10.021488 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c65da4-9045-4bcb-b803-e202caa3b8c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:10.023581 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c65da4-9045-4bcb-b803-e202caa3b8c7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:10.048078 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 367/753] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:10.048078 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c65da4-9045-4bcb-b803-e202caa3b8c7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:10.063443 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 387/754] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:10.064512 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24b900-9a2e-46e5-8800-6834ad397106 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:10.150974 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:10.195420 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24b900-9a2e-46e5-8800-6834ad397106 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:10.196039 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c65da4-9045-4bcb-b803-e202caa3b8c7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:10.232770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:10.237568 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa1b03-b323-4828-bb7e-2e28221263b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:10.238236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa1b03-b323-4828-bb7e-2e28221263b6 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:10.238507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa1b03-b323-4828-bb7e-2e28221263b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:10.238783 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa1b03-b323-4828-bb7e-2e28221263b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:10.239628 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa1b03-b323-4828-bb7e-2e28221263b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:10.239959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa1b03-b323-4828-bb7e-2e28221263b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:10.241483 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa1b03-b323-4828-bb7e-2e28221263b6 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:10.242870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6001c67-dd16-4cc2-a694-7cc10dd355b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:10.243783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6001c67-dd16-4cc2-a694-7cc10dd355b2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:10.244071 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6001c67-dd16-4cc2-a694-7cc10dd355b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:10.244337 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6001c67-dd16-4cc2-a694-7cc10dd355b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:10.245596 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6001c67-dd16-4cc2-a694-7cc10dd355b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:10.245896 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6001c67-dd16-4cc2-a694-7cc10dd355b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:10.247288 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6001c67-dd16-4cc2-a694-7cc10dd355b2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:10.255028 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 388/755] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:47:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:10.256294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa1b03-b323-4828-bb7e-2e28221263b6 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:10.264228 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 368/756] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:10.265257 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6001c67-dd16-4cc2-a694-7cc10dd355b2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:10.378671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa1b03-b323-4828-bb7e-2e28221263b6 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:10.382554 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6001c67-dd16-4cc2-a694-7cc10dd355b2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:10.404081 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:10.416800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf4268-cbde-425e-a5f4-d18399874ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:10.417459 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf4268-cbde-425e-a5f4-d18399874ccc None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:10.417741 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf4268-cbde-425e-a5f4-d18399874ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:10.418013 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf4268-cbde-425e-a5f4-d18399874ccc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:10.422529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43695e4c-87b4-42c4-ae35-abaad808dbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:10.423188 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43695e4c-87b4-42c4-ae35-abaad808dbce None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:10.423468 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43695e4c-87b4-42c4-ae35-abaad808dbce None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:10.423779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43695e4c-87b4-42c4-ae35-abaad808dbce None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:10.424412 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43695e4c-87b4-42c4-ae35-abaad808dbce None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:10.425788 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43695e4c-87b4-42c4-ae35-abaad808dbce None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:10.426397 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf4268-cbde-425e-a5f4-d18399874ccc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:10.427879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf4268-cbde-425e-a5f4-d18399874ccc None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:10.435254 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-43695e4c-87b4-42c4-ae35-abaad808dbce None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 04:47:10.435254 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:10.435254 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 04:47:10.435254 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 04:47:10.435254 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 04:47:10.435254 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 04:47:10.435254 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 04:47:10.435254 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:10.435254 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 04:47:10.435254 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:10.435254 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:47:10.435254 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:10.435254 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:10.435254 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:10.435254 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:10.435254 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:10.436238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:10.436238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:10.436238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:10.436238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:10.436238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:10.436238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:10.436238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:10.436238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 04:47:10.436238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 04:47:10.436238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 04:47:10.436238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 04:47:10.436238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:10.436238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:10.436238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:10.436238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:10.436238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:10.437073 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:10.437073 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:10.437073 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:10.437073 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:10.437073 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:10.437073 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:10.437073 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:10.437073 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:10.437073 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:10.437073 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:10.437073 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:10.437073 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 04:47:10.437073 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:10.437073 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 04:47:10.437073 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:10.437073 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 369/757] 10.222.0.17 () {56 vars in 1129 bytes} [Tue Jul 26 04:47:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:10.437921 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43695e4c-87b4-42c4-ae35-abaad808dbce None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:10.439939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-dacf4268-cbde-425e-a5f4-d18399874ccc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 04:47:10.439939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:10.439939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:10.439939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:10.439939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:10.439939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:10.439939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:10.439939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:10.439939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:10.439939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:10.439939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:10.439939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:10.439939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:47:10.439939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:47:10.439939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:47:10.439939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:47:10.440843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:10.440843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:10.440843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:47:10.440843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:10.440843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:47:10.440843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:10.440843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:10.440843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:10.440843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:10.440843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:10.440843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:10.440843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:10.440843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:10.440843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:10.440843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:10.440843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:10.440843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:10.440843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:10.442573 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:10.442573 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:10.442573 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:47:10.442573 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:47:10.442573 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:47:10.442573 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:47:10.442573 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:47:10.442573 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:47:10.442573 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 04:47:10.442573 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:10.445966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 389/758] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Jul 26 04:47:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:10.446868 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf4268-cbde-425e-a5f4-d18399874ccc None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:10.495987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:10.547930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43695e4c-87b4-42c4-ae35-abaad808dbce None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:10.553270 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf4268-cbde-425e-a5f4-d18399874ccc None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:10.586817 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e5c4de-43d5-4e98-9028-a1e3fb2d82e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:10.587708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5c4de-43d5-4e98-9028-a1e3fb2d82e0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:10.587998 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5c4de-43d5-4e98-9028-a1e3fb2d82e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:10.588284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5c4de-43d5-4e98-9028-a1e3fb2d82e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:10.589142 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e5c4de-43d5-4e98-9028-a1e3fb2d82e0 None admin] RBAC: Adding query filter params (role.id=72bf8bd23d2c4b76ad32b7b35d6db412, scope.project.id=d583e0be8ef74766bffd6c3c07af18f1, user.id=d5218319aaa5424882714bffcbeee730) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:10.589459 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e5c4de-43d5-4e98-9028-a1e3fb2d82e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:10.591276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e5c4de-43d5-4e98-9028-a1e3fb2d82e0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:10.592659 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ba406-1169-4b7e-913a-57d9b98f076c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:10.593482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ba406-1169-4b7e-913a-57d9b98f076c None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:10.593907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ba406-1169-4b7e-913a-57d9b98f076c None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:10.594368 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ba406-1169-4b7e-913a-57d9b98f076c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:10.597545 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 370/759] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:47:10 2022] GET /identity/v3/role_assignments?user.id=d5218319aaa5424882714bffcbeee730&scope.project.id=d583e0be8ef74766bffd6c3c07af18f1&role.id=72bf8bd23d2c4b76ad32b7b35d6db412 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:10.598565 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e5c4de-43d5-4e98-9028-a1e3fb2d82e0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:10.601803 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ba406-1169-4b7e-913a-57d9b98f076c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:10.603313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ba406-1169-4b7e-913a-57d9b98f076c None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:10.624759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-be6ba406-1169-4b7e-913a-57d9b98f076c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:10.624759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:10.624759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:10.624759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:10.624759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:10.624759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:10.624759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:10.624759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:10.624759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:10.624759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:10.624759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:10.624759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:10.624759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:47:10.624759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:47:10.624759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:47:10.625716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:47:10.625716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:10.625716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:10.625716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:10.625716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:10.625716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:10.625716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:10.625716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:10.625716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:10.625716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:10.625716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:10.625716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:10.625716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:10.625716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:10.625716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:10.625716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:10.625716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:10.627243 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:47:10.627243 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:47:10.627243 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:47:10.627243 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:47:10.627243 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:47:10.627243 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:47:10.627243 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:10.627243 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:10.627243 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 390/760] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:47:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:10.632134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ba406-1169-4b7e-913a-57d9b98f076c None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:10.683400 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:10.755430 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ba406-1169-4b7e-913a-57d9b98f076c None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:10.756880 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e5c4de-43d5-4e98-9028-a1e3fb2d82e0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:10.782991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:10.799782 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de348c14-111c-456f-9544-ccfd53c5282a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:10.800779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de348c14-111c-456f-9544-ccfd53c5282a None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:10.801209 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de348c14-111c-456f-9544-ccfd53c5282a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:10.801533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de348c14-111c-456f-9544-ccfd53c5282a None admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:10.802356 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de348c14-111c-456f-9544-ccfd53c5282a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:10.802660 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de348c14-111c-456f-9544-ccfd53c5282a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:10.804592 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de348c14-111c-456f-9544-ccfd53c5282a None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:10.810101 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01070112-ca29-4722-b777-90e9fb96ed87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:10.810762 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01070112-ca29-4722-b777-90e9fb96ed87 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:10.811052 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01070112-ca29-4722-b777-90e9fb96ed87 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:10.811387 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01070112-ca29-4722-b777-90e9fb96ed87 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:10.812086 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01070112-ca29-4722-b777-90e9fb96ed87 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:10.812395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01070112-ca29-4722-b777-90e9fb96ed87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:10.813794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01070112-ca29-4722-b777-90e9fb96ed87 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:10.819477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 371/761] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Jul 26 04:47:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:10.820406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01070112-ca29-4722-b777-90e9fb96ed87 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:10.831403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 391/762] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Jul 26 04:47:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:10.832333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de348c14-111c-456f-9544-ccfd53c5282a None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:10.936130 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01070112-ca29-4722-b777-90e9fb96ed87 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:10.942112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:10.945656 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de348c14-111c-456f-9544-ccfd53c5282a None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:10.976458 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc52834-25b6-495d-a242-8e2b6556d7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:10.977132 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc52834-25b6-495d-a242-8e2b6556d7dc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:10.977418 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc52834-25b6-495d-a242-8e2b6556d7dc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:10.977703 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc52834-25b6-495d-a242-8e2b6556d7dc None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:10.978470 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc52834-25b6-495d-a242-8e2b6556d7dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:10.978781 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc52834-25b6-495d-a242-8e2b6556d7dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:10.980352 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc52834-25b6-495d-a242-8e2b6556d7dc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:10.989407 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34312003-ff03-4620-a0c3-c2b801400a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:10.990267 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34312003-ff03-4620-a0c3-c2b801400a22 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:10.990771 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34312003-ff03-4620-a0c3-c2b801400a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:10.991188 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34312003-ff03-4620-a0c3-c2b801400a22 None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:10.992051 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34312003-ff03-4620-a0c3-c2b801400a22 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:10.992499 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34312003-ff03-4620-a0c3-c2b801400a22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:10.993814 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 372/763] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:47:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:10.994422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34312003-ff03-4620-a0c3-c2b801400a22 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:10.994690 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc52834-25b6-495d-a242-8e2b6556d7dc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:11.001918 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 392/764] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Jul 26 04:47:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:11.003022 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34312003-ff03-4620-a0c3-c2b801400a22 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:11.046963 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:11.113837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc52834-25b6-495d-a242-8e2b6556d7dc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:11.141333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34312003-ff03-4620-a0c3-c2b801400a22 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:11.162755 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ea664-22db-41b4-802a-1d359b8d2c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:11.163705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ea664-22db-41b4-802a-1d359b8d2c90 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:11.163705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ea664-22db-41b4-802a-1d359b8d2c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:11.163990 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ea664-22db-41b4-802a-1d359b8d2c90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:11.180638 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e76337-b6f1-43d2-ae91-39277710f252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:11.181695 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e76337-b6f1-43d2-ae91-39277710f252 None admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:11.182157 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e76337-b6f1-43d2-ae91-39277710f252 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:11.182670 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e76337-b6f1-43d2-ae91-39277710f252 None admin] PATH_INFO: `/v3/projects/d583e0be8ef74766bffd6c3c07af18f1/users/2faeed9002ee4154bc980c146cd06bcc/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:11.184939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e76337-b6f1-43d2-ae91-39277710f252 None admin] RBAC: Authorizing `identity:create_grant(project_id=d583e0be8ef74766bffd6c3c07af18f1, user_id=2faeed9002ee4154bc980c146cd06bcc, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:11.186668 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e76337-b6f1-43d2-ae91-39277710f252 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:11.204312 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 393/765] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jul 26 04:47:11 2022] PUT /identity/v3/projects/d583e0be8ef74766bffd6c3c07af18f1/users/2faeed9002ee4154bc980c146cd06bcc/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:47:11.206015 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e76337-b6f1-43d2-ae91-39277710f252 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:11.246299 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:11.272286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ea664-22db-41b4-802a-1d359b8d2c90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:11.276213 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ea664-22db-41b4-802a-1d359b8d2c90 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:11.373141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:11.432906 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e76337-b6f1-43d2-ae91-39277710f252 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:11.454897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 373/766] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:47:10 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 04:47:11.464544 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ea664-22db-41b4-802a-1d359b8d2c90 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:11.489990 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae256f-0807-4d2f-a581-4523629c447e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:11.490644 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae256f-0807-4d2f-a581-4523629c447e None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:11.490931 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae256f-0807-4d2f-a581-4523629c447e None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:11.491208 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae256f-0807-4d2f-a581-4523629c447e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:11.492049 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae256f-0807-4d2f-a581-4523629c447e None admin] RBAC: Adding query filter params (interface=public, service_id=e0af1a0dfbfb4d6a9c4196f0f2f2d1c8, region_id=RegionOne) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:11.492455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae256f-0807-4d2f-a581-4523629c447e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:11.494150 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae256f-0807-4d2f-a581-4523629c447e None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:11.516441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 394/767] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Jul 26 04:47:11 2022] GET /identity/v3/endpoints?service_id=e0af1a0dfbfb4d6a9c4196f0f2f2d1c8&interface=public®ion_id=RegionOne => generated 194 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:11.522020 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae256f-0807-4d2f-a581-4523629c447e None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:11.642052 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ea664-22db-41b4-802a-1d359b8d2c90 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:11.700095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:11.713738 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647f9e4-4cc6-4ec6-bdc0-8ff42fae244f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:11.714745 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647f9e4-4cc6-4ec6-bdc0-8ff42fae244f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:11.715268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647f9e4-4cc6-4ec6-bdc0-8ff42fae244f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:11.716161 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647f9e4-4cc6-4ec6-bdc0-8ff42fae244f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:11.727667 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647f9e4-4cc6-4ec6-bdc0-8ff42fae244f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:11.729639 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647f9e4-4cc6-4ec6-bdc0-8ff42fae244f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:11.731858 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:11.737763 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae256f-0807-4d2f-a581-4523629c447e None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:11.747823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-8647f9e4-4cc6-4ec6-bdc0-8ff42fae244f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:11.747823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:11.747823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:11.747823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:11.747823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:11.747823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:11.747823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:11.747823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:11.747823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:11.747823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:11.747823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:11.747823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:11.747823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:11.747823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:11.747823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:11.748771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:11.748771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:11.748771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:11.748771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:11.748771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:11.748771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:11.748771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:11.748771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:11.748771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:11.748771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:11.748771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:11.748771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:11.748771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:11.748771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:11.748771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:11.748771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:11.748771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:11.749658 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:11.749658 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:11.749658 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:11.749658 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:11.752060 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 374/768] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:11.760894 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647f9e4-4cc6-4ec6-bdc0-8ff42fae244f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:11.804486 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c750f0c-9a61-4eca-be3b-82028ab76848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:11.805214 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c750f0c-9a61-4eca-be3b-82028ab76848 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:11.805477 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c750f0c-9a61-4eca-be3b-82028ab76848 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:11.805758 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c750f0c-9a61-4eca-be3b-82028ab76848 None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:11.806434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c750f0c-9a61-4eca-be3b-82028ab76848 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:11.808005 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c750f0c-9a61-4eca-be3b-82028ab76848 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:11.813999 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 395/769] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Jul 26 04:47:11 2022] GET /identity/v3/services => generated 772 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:11.815354 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ec3fc-8355-4c32-97a9-ca222a3a1b8c None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:11.815719 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ec3fc-8355-4c32-97a9-ca222a3a1b8c None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:11.816006 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ec3fc-8355-4c32-97a9-ca222a3a1b8c None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:11.817648 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 396/770] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:11.831189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3de5f-4187-4860-9942-cd95b3ea6598 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:11.831544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3de5f-4187-4860-9942-cd95b3ea6598 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:11.831794 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3de5f-4187-4860-9942-cd95b3ea6598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:11.858763 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-b5e3de5f-4187-4860-9942-cd95b3ea6598 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:11.860173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-b5e3de5f-4187-4860-9942-cd95b3ea6598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:11.910726 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647f9e4-4cc6-4ec6-bdc0-8ff42fae244f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:11.986331 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca35f0-7450-4445-9bd7-5c20427c3389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:11.986895 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca35f0-7450-4445-9bd7-5c20427c3389 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:11.987178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca35f0-7450-4445-9bd7-5c20427c3389 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:11.987454 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca35f0-7450-4445-9bd7-5c20427c3389 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:11.988203 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca35f0-7450-4445-9bd7-5c20427c3389 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:11.988508 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca35f0-7450-4445-9bd7-5c20427c3389 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:11.989929 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca35f0-7450-4445-9bd7-5c20427c3389 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:11.999828 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 397/771] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:12.017764 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 375/772] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:12.023503 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3de5f-4187-4860-9942-cd95b3ea6598 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:12.041742 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca35f0-7450-4445-9bd7-5c20427c3389 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:12.111382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:12.168152 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:12.186361 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca35f0-7450-4445-9bd7-5c20427c3389 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:12.203006 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3de5f-4187-4860-9942-cd95b3ea6598 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:12.253713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295bc14a-5114-4586-b2da-8fd3be59f248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:12.254302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295bc14a-5114-4586-b2da-8fd3be59f248 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:12.254588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295bc14a-5114-4586-b2da-8fd3be59f248 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:12.254864 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295bc14a-5114-4586-b2da-8fd3be59f248 None admin] PATH_INFO: `/v3/services/network` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:12.255503 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295bc14a-5114-4586-b2da-8fd3be59f248 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:12.256912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295bc14a-5114-4586-b2da-8fd3be59f248 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:12.270187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-295bc14a-5114-4586-b2da-8fd3be59f248 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 04:47:12.270187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:12.270187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 04:47:12.270187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 04:47:12.270187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 04:47:12.270187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 04:47:12.270187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 04:47:12.270187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:12.270187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 04:47:12.270187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:12.270187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:47:12.270187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:12.270187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:12.270187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:12.270187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:12.270187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:12.270187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:12.271246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:12.271246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:12.271246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:12.271246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:12.271246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:12.271246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:12.271246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 04:47:12.271246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 04:47:12.271246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 04:47:12.271246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 04:47:12.271246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:12.271246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:12.271246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:12.271246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:12.271246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:12.271246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:12.272126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:12.272126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:12.272126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:12.272126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:12.272126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:12.272126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:12.272126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:12.272126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:12.272126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:12.272126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:12.272126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 04:47:12.272126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:12.272126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 04:47:12.272126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:12.272126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 376/773] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:47:12 2022] GET /identity/v3/services/network => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:47:12.276848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfbbc4-e28b-46d3-90e8-b98f9a0a61f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:12.277767 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfbbc4-e28b-46d3-90e8-b98f9a0a61f3 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:12.278270 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfbbc4-e28b-46d3-90e8-b98f9a0a61f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:12.278739 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfbbc4-e28b-46d3-90e8-b98f9a0a61f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:12.280522 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295bc14a-5114-4586-b2da-8fd3be59f248 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:12.300841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bfbbc4-e28b-46d3-90e8-b98f9a0a61f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:12.302848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bfbbc4-e28b-46d3-90e8-b98f9a0a61f3 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:12.323311 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-33bfbbc4-e28b-46d3-90e8-b98f9a0a61f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:12.323311 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:12.323311 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:12.323311 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:12.323311 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:12.323311 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:12.323311 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:12.323311 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:12.323311 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:12.323311 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:12.323311 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:12.323311 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:12.323311 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:47:12.323311 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:47:12.323311 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:47:12.324315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:47:12.324315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:12.324315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:12.324315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:12.324315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:12.324315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:12.324315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:12.324315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:12.324315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:12.324315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:12.324315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:12.324315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:12.324315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:12.324315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:12.324315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:12.324315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:12.324315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:12.325193 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:47:12.325193 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:47:12.325193 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:47:12.325193 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:47:12.325193 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:47:12.325193 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:47:12.325193 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:12.325193 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:12.331773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 398/774] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:47:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:12.343581 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfbbc4-e28b-46d3-90e8-b98f9a0a61f3 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:12.502866 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295bc14a-5114-4586-b2da-8fd3be59f248 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:12.533348 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:12.554327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfbbc4-e28b-46d3-90e8-b98f9a0a61f3 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:12.570518 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40ab5b-bfb9-4f44-9993-3818bced508e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:12.571164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40ab5b-bfb9-4f44-9993-3818bced508e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:12.571451 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40ab5b-bfb9-4f44-9993-3818bced508e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:12.571753 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40ab5b-bfb9-4f44-9993-3818bced508e None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:12.572448 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40ab5b-bfb9-4f44-9993-3818bced508e None admin] RBAC: Adding query filter params (name=network) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:12.572760 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40ab5b-bfb9-4f44-9993-3818bced508e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:12.574251 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40ab5b-bfb9-4f44-9993-3818bced508e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:12.585493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 377/775] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Jul 26 04:47:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:12.591887 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40ab5b-bfb9-4f44-9993-3818bced508e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:12.640962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcd34b-a09f-4f4b-881f-ecf9ee8ea654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:12.641925 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcd34b-a09f-4f4b-881f-ecf9ee8ea654 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:12.642408 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcd34b-a09f-4f4b-881f-ecf9ee8ea654 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:12.642879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcd34b-a09f-4f4b-881f-ecf9ee8ea654 None admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:12.643854 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbcd34b-a09f-4f4b-881f-ecf9ee8ea654 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:12.644319 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbcd34b-a09f-4f4b-881f-ecf9ee8ea654 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:12.645943 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbcd34b-a09f-4f4b-881f-ecf9ee8ea654 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:12.672085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 399/776] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:47:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:12.680608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcd34b-a09f-4f4b-881f-ecf9ee8ea654 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:12.691578 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:12.713244 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40ab5b-bfb9-4f44-9993-3818bced508e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:12.769311 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b633ab-c21d-4833-ac62-9e1c8af86d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:12.769959 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b633ab-c21d-4833-ac62-9e1c8af86d14 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:12.770246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b633ab-c21d-4833-ac62-9e1c8af86d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:12.770522 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b633ab-c21d-4833-ac62-9e1c8af86d14 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:12.771220 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b633ab-c21d-4833-ac62-9e1c8af86d14 None admin] RBAC: Adding query filter params (type=network) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:12.771528 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b633ab-c21d-4833-ac62-9e1c8af86d14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:12.772949 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b633ab-c21d-4833-ac62-9e1c8af86d14 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:12.786861 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 378/777] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Jul 26 04:47:12 2022] GET /identity/v3/services?type=network => generated 350 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:12.789325 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b633ab-c21d-4833-ac62-9e1c8af86d14 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:12.845185 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:12.888046 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcd34b-a09f-4f4b-881f-ecf9ee8ea654 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:12.944021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b633ab-c21d-4833-ac62-9e1c8af86d14 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:12.949579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe37632-5591-4958-9256-267feb028d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:12.950227 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe37632-5591-4958-9256-267feb028d51 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:12.950521 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe37632-5591-4958-9256-267feb028d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:12.950798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe37632-5591-4958-9256-267feb028d51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:12.951634 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe37632-5591-4958-9256-267feb028d51 None admin] RBAC: Adding query filter params (role.id=e5048f4e7c6b4494a15d628758090380, scope.project.id=d583e0be8ef74766bffd6c3c07af18f1, user.id=3a8f3eb811ae40ad9cb64261f95a992e) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:12.952527 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe37632-5591-4958-9256-267feb028d51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:12.954020 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe37632-5591-4958-9256-267feb028d51 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:12.962468 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 400/778] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:47:12 2022] GET /identity/v3/role_assignments?user.id=3a8f3eb811ae40ad9cb64261f95a992e&scope.project.id=d583e0be8ef74766bffd6c3c07af18f1&role.id=e5048f4e7c6b4494a15d628758090380 => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:12.963876 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06037ae9-6713-450f-8008-05feb49e5501 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:12.964166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06037ae9-6713-450f-8008-05feb49e5501 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:12.964447 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06037ae9-6713-450f-8008-05feb49e5501 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:12.971053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 401/779] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:12.972668 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8881c9a-a516-45a2-a884-77afc8ca4072 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:12.972965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8881c9a-a516-45a2-a884-77afc8ca4072 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:12.973249 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8881c9a-a516-45a2-a884-77afc8ca4072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:12.996244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-f8881c9a-a516-45a2-a884-77afc8ca4072 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:12.997610 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-f8881c9a-a516-45a2-a884-77afc8ca4072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:13.001965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60698610-8981-4d64-a48a-c2d47f9651ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:13.002727 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60698610-8981-4d64-a48a-c2d47f9651ad None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:13.003016 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60698610-8981-4d64-a48a-c2d47f9651ad None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:13.003303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60698610-8981-4d64-a48a-c2d47f9651ad None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:13.004158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60698610-8981-4d64-a48a-c2d47f9651ad None admin] RBAC: Adding query filter params (interface=public, service_id=025581d59d7d4a97bb4c0db31e4d871c, region_id=RegionOne) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:13.004478 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60698610-8981-4d64-a48a-c2d47f9651ad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:13.005935 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60698610-8981-4d64-a48a-c2d47f9651ad None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:13.027910 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 379/780] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Jul 26 04:47:12 2022] GET /identity/v3/endpoints?service_id=025581d59d7d4a97bb4c0db31e4d871c&interface=public®ion_id=RegionOne => generated 194 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:13.034599 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60698610-8981-4d64-a48a-c2d47f9651ad None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:13.084571 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:13.107806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 402/781] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:13.146869 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8881c9a-a516-45a2-a884-77afc8ca4072 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:13.234466 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60698610-8981-4d64-a48a-c2d47f9651ad None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:13.250924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:13.284666 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc36925-2077-4f76-a6e0-e24f22cfde2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:13.285949 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc36925-2077-4f76-a6e0-e24f22cfde2d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:13.286241 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc36925-2077-4f76-a6e0-e24f22cfde2d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:13.286520 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc36925-2077-4f76-a6e0-e24f22cfde2d None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:13.287670 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc36925-2077-4f76-a6e0-e24f22cfde2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:13.289550 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc36925-2077-4f76-a6e0-e24f22cfde2d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:13.296347 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 380/782] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Jul 26 04:47:13 2022] GET /identity/v3/services => generated 772 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:13.344566 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8881c9a-a516-45a2-a884-77afc8ca4072 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:13.403830 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29195680-6162-4afb-9d68-af9f1eb0d272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:13.404507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29195680-6162-4afb-9d68-af9f1eb0d272 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:13.404786 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29195680-6162-4afb-9d68-af9f1eb0d272 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:13.405055 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29195680-6162-4afb-9d68-af9f1eb0d272 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:13.424787 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29195680-6162-4afb-9d68-af9f1eb0d272 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:47:13.429831 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-29195680-6162-4afb-9d68-af9f1eb0d272 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:47:13.429831 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:13.429831 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:13.429831 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:13.429831 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:13.429831 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:13.429831 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:13.429831 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:13.429831 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:13.429831 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:13.429831 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:13.429831 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:13.429831 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:47:13.429831 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:47:13.429831 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:47:13.429831 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:47:13.430772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:47:13.430772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:47:13.430772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:47:13.430772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:47:13.430772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:13.430772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:13.430772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:13.430772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:13.430772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:13.430772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:13.430772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:13.430772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:13.430772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:13.430772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:13.430772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:13.430772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:13.431660 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:13.431660 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:13.431660 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:13.431660 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:13.431660 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:47:13.431660 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:47:13.431660 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:47:13.431660 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:47:13.431660 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:47:13.431660 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:47:13.431660 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:47:13.431660 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:13.431660 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 403/783] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:47:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:13.437297 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc36925-2077-4f76-a6e0-e24f22cfde2d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:13.582634 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b40d1f-b582-4799-b6de-feb064df19b5 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:13.583025 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b40d1f-b582-4799-b6de-feb064df19b5 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:13.583195 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b40d1f-b582-4799-b6de-feb064df19b5 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:13.584985 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 404/784] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:47:13.592467 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e1a96-6572-4c35-9ad1-15caa01b0601 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:13.592773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e1a96-6572-4c35-9ad1-15caa01b0601 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:13.593026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e1a96-6572-4c35-9ad1-15caa01b0601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:13.616142 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-315e1a96-6572-4c35-9ad1-15caa01b0601 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:13.618220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-315e1a96-6572-4c35-9ad1-15caa01b0601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:13.620503 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc36925-2077-4f76-a6e0-e24f22cfde2d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:13.698669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:13.703650 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d6988-b1c8-4c3b-859a-d39ddd4b67b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:13.704951 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d6988-b1c8-4c3b-859a-d39ddd4b67b6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:13.705470 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d6988-b1c8-4c3b-859a-d39ddd4b67b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:13.706120 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d6988-b1c8-4c3b-859a-d39ddd4b67b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:13.708452 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d6988-b1c8-4c3b-859a-d39ddd4b67b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:13.708976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d6988-b1c8-4c3b-859a-d39ddd4b67b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:13.711414 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d6988-b1c8-4c3b-859a-d39ddd4b67b6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:13.719691 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 405/785] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 04:47:13.722570 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 381/786] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:47:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:13.730575 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e1a96-6572-4c35-9ad1-15caa01b0601 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:13.743356 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d6988-b1c8-4c3b-859a-d39ddd4b67b6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:13.795941 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:13.899225 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d6988-b1c8-4c3b-859a-d39ddd4b67b6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:13.904652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e1a96-6572-4c35-9ad1-15caa01b0601 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:13.963421 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da64676-ff37-43d3-8665-a3760a6c2299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:13.964136 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da64676-ff37-43d3-8665-a3760a6c2299 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:13.964392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da64676-ff37-43d3-8665-a3760a6c2299 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:13.964663 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da64676-ff37-43d3-8665-a3760a6c2299 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:13.973994 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881a4fe-f618-486c-9e74-d6f2f6fe6293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:13.974655 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881a4fe-f618-486c-9e74-d6f2f6fe6293 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:13.974935 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881a4fe-f618-486c-9e74-d6f2f6fe6293 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:13.975209 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881a4fe-f618-486c-9e74-d6f2f6fe6293 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:13.991990 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da64676-ff37-43d3-8665-a3760a6c2299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:47:14.000661 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881a4fe-f618-486c-9e74-d6f2f6fe6293 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:14.003338 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-6da64676-ff37-43d3-8665-a3760a6c2299 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:47:14.003338 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:14.003338 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:14.003338 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:14.003338 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:14.003338 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:14.003338 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:14.003338 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:14.003338 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:14.003338 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:14.003338 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:14.003338 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:14.003338 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:47:14.003338 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:47:14.003338 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:47:14.003338 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:47:14.004296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:47:14.004296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:47:14.004296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:47:14.004296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:47:14.004296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:14.004296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:14.004296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:14.004296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:14.004296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:14.004296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:14.004296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:14.004296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:14.004296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:14.004296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:14.004296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:14.004296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:14.005121 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:14.005121 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:14.005121 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:14.005121 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:14.005121 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:47:14.005121 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:47:14.005121 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:47:14.005121 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:47:14.005121 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:47:14.005121 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:47:14.005121 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:47:14.005121 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:14.005121 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 382/787] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:47:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:47:14.006461 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881a4fe-f618-486c-9e74-d6f2f6fe6293 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:14.012650 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da64676-ff37-43d3-8665-a3760a6c2299 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:14.023410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-4881a4fe-f618-486c-9e74-d6f2f6fe6293 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:14.023410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:14.023410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:14.023410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:14.023410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:14.023410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:14.023410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:14.023410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:14.023410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:14.023410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:14.023410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:14.023410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:14.023410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:14.023410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:14.023410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:14.025339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:14.025339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:14.025339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:14.025339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:14.025339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:14.025339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:14.025339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:14.025339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:14.025339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:14.025339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:14.025339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:14.025339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:14.025339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:14.025339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:14.025339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:14.025339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:14.025339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:14.026369 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:14.026369 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:14.026369 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:14.026369 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:14.030212 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 406/788] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:14.039133 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881a4fe-f618-486c-9e74-d6f2f6fe6293 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:14.112739 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:14.161503 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da64676-ff37-43d3-8665-a3760a6c2299 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:14.178331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881a4fe-f618-486c-9e74-d6f2f6fe6293 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:14.184145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:14.197578 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e6771-2dba-49e3-810d-f8eb7a2cc033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:14.198606 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e6771-2dba-49e3-810d-f8eb7a2cc033 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:14.199088 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e6771-2dba-49e3-810d-f8eb7a2cc033 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:14.199605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e6771-2dba-49e3-810d-f8eb7a2cc033 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:14.200537 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e6771-2dba-49e3-810d-f8eb7a2cc033 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:14.201038 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e6771-2dba-49e3-810d-f8eb7a2cc033 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:14.202648 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e6771-2dba-49e3-810d-f8eb7a2cc033 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:14.212356 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 383/789] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:47:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:14.214269 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c78e2-2d76-4802-81d7-8d8fb76ee990 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:14.214752 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913d9e3-9cea-40f0-8169-d4016d831cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:14.214977 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c78e2-2d76-4802-81d7-8d8fb76ee990 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:14.215387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913d9e3-9cea-40f0-8169-d4016d831cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:14.216064 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913d9e3-9cea-40f0-8169-d4016d831cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:14.216346 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913d9e3-9cea-40f0-8169-d4016d831cd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:14.216460 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c78e2-2d76-4802-81d7-8d8fb76ee990 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:14.217393 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913d9e3-9cea-40f0-8169-d4016d831cd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:14.217707 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913d9e3-9cea-40f0-8169-d4016d831cd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:14.218705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 384/790] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:14.219632 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c78e2-2d76-4802-81d7-8d8fb76ee990 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:14.223119 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913d9e3-9cea-40f0-8169-d4016d831cd5 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:14.236260 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 407/791] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:14.240922 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0641750-a43c-4fa7-8471-2960ca3422c6 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:14.240922 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0641750-a43c-4fa7-8471-2960ca3422c6 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:14.240922 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0641750-a43c-4fa7-8471-2960ca3422c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:14.258714 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-c0641750-a43c-4fa7-8471-2960ca3422c6 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:14.260086 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c0641750-a43c-4fa7-8471-2960ca3422c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:14.330397 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c78e2-2d76-4802-81d7-8d8fb76ee990 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:14.332448 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 408/792] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:14.339061 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0641750-a43c-4fa7-8471-2960ca3422c6 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:14.374843 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d88bb-7c20-4d20-9239-cb9850201c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:14.375528 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d88bb-7c20-4d20-9239-cb9850201c3e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:14.375847 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d88bb-7c20-4d20-9239-cb9850201c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:14.376274 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d88bb-7c20-4d20-9239-cb9850201c3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:14.392875 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:14.395145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670d88bb-7c20-4d20-9239-cb9850201c3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:14.396811 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670d88bb-7c20-4d20-9239-cb9850201c3e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:14.412642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-670d88bb-7c20-4d20-9239-cb9850201c3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:14.412642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:14.412642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:14.412642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:14.412642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:14.412642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:14.412642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:14.412642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:14.412642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:14.412642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:14.412642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:14.412642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:14.412642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:14.412642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:14.412642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:14.413561 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:14.413561 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:14.413561 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:14.413561 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:14.413561 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:14.413561 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:14.413561 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:14.413561 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:14.413561 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:14.413561 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:14.413561 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:14.413561 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:14.413561 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:14.413561 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:14.413561 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:14.413561 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:14.413561 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:14.415096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:14.415096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:14.415096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:14.415096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:14.415096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 385/793] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:14.415096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d88bb-7c20-4d20-9239-cb9850201c3e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:14.426693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:14.478132 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0641750-a43c-4fa7-8471-2960ca3422c6 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:14.515031 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172031ba-76ca-46b0-b077-f326acb3a247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:14.516216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172031ba-76ca-46b0-b077-f326acb3a247 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:14.516705 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172031ba-76ca-46b0-b077-f326acb3a247 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:14.517212 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172031ba-76ca-46b0-b077-f326acb3a247 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:14.518895 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d88bb-7c20-4d20-9239-cb9850201c3e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:14.525480 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172031ba-76ca-46b0-b077-f326acb3a247 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:14.527363 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172031ba-76ca-46b0-b077-f326acb3a247 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:14.541930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-172031ba-76ca-46b0-b077-f326acb3a247 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 04:47:14.541930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:14.541930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:14.541930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:14.541930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:14.541930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:14.541930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:14.541930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:14.541930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:14.541930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:14.541930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:14.541930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:14.541930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:47:14.541930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:47:14.541930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:47:14.541930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:47:14.542966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:14.542966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:14.542966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:47:14.542966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:14.542966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:47:14.542966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:14.542966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:14.542966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:14.542966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:14.542966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:14.542966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:14.542966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:14.542966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:14.542966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:14.542966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:14.542966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:14.542966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:14.542966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:14.543953 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:14.543953 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:14.543953 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:47:14.543953 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:47:14.543953 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:47:14.543953 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:47:14.543953 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:47:14.543953 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:47:14.543953 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 04:47:14.543953 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:14.546788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 409/794] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:47:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:14.547962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172031ba-76ca-46b0-b077-f326acb3a247 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:14.558763 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4002d08-737e-4c9b-b644-60b807386ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:14.559422 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4002d08-737e-4c9b-b644-60b807386ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:14.559753 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4002d08-737e-4c9b-b644-60b807386ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:14.560031 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4002d08-737e-4c9b-b644-60b807386ef0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:14.560666 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4002d08-737e-4c9b-b644-60b807386ef0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:14.562058 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4002d08-737e-4c9b-b644-60b807386ef0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:14.571392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d4002d08-737e-4c9b-b644-60b807386ef0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 04:47:14.571392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:14.571392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 04:47:14.571392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 04:47:14.571392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 04:47:14.571392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 04:47:14.571392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 04:47:14.571392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:14.571392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 04:47:14.571392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:14.571392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:47:14.571392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:14.571392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:14.571392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:14.571392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:14.571392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:14.572393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:14.572393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:14.572393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:14.572393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:14.572393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:14.572393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:14.572393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:14.572393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 04:47:14.572393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 04:47:14.572393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 04:47:14.572393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 04:47:14.572393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:14.572393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:14.572393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:14.572393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:14.572393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:14.578516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:14.578516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:14.578516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:14.578516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:14.578516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:14.578516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:14.578516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:14.578516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:14.578516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:14.578516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:14.578516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:14.578516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 04:47:14.578516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:14.578516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 04:47:14.578516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:14.578516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 386/795] 10.222.0.17 () {56 vars in 1129 bytes} [Tue Jul 26 04:47:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:14.579395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4002d08-737e-4c9b-b644-60b807386ef0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:14.646554 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:14.660973 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:14.676310 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172031ba-76ca-46b0-b077-f326acb3a247 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:14.678239 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4002d08-737e-4c9b-b644-60b807386ef0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:14.713624 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a3b8f-dd5b-4207-a515-71acbe28d575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:14.714355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a3b8f-dd5b-4207-a515-71acbe28d575 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:14.714643 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a3b8f-dd5b-4207-a515-71acbe28d575 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:14.714919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a3b8f-dd5b-4207-a515-71acbe28d575 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:14.716194 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a3b8f-dd5b-4207-a515-71acbe28d575 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:14.716498 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a3b8f-dd5b-4207-a515-71acbe28d575 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:14.718051 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a3b8f-dd5b-4207-a515-71acbe28d575 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:14.718369 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992dab0-a872-4906-9a0a-90a5399ed223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:14.719231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992dab0-a872-4906-9a0a-90a5399ed223 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:14.719844 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992dab0-a872-4906-9a0a-90a5399ed223 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:14.720319 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992dab0-a872-4906-9a0a-90a5399ed223 None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:14.721355 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992dab0-a872-4906-9a0a-90a5399ed223 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:14.721866 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992dab0-a872-4906-9a0a-90a5399ed223 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:14.723466 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992dab0-a872-4906-9a0a-90a5399ed223 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:14.736896 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 387/796] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Jul 26 04:47:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:14.737758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a3b8f-dd5b-4207-a515-71acbe28d575 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:14.738507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 410/797] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:14.743701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992dab0-a872-4906-9a0a-90a5399ed223 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:14.847540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a3b8f-dd5b-4207-a515-71acbe28d575 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:14.860744 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992dab0-a872-4906-9a0a-90a5399ed223 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:14.877311 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:14.883394 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c9671-06f7-4ccb-bd1b-67489d9ec272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:14.884409 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c9671-06f7-4ccb-bd1b-67489d9ec272 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:14.885035 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c9671-06f7-4ccb-bd1b-67489d9ec272 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:14.885448 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c9671-06f7-4ccb-bd1b-67489d9ec272 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:14.886314 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c9671-06f7-4ccb-bd1b-67489d9ec272 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:14.886720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c9671-06f7-4ccb-bd1b-67489d9ec272 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:14.888409 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c9671-06f7-4ccb-bd1b-67489d9ec272 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:14.893569 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 388/798] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Jul 26 04:47:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:14.894678 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c9671-06f7-4ccb-bd1b-67489d9ec272 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:14.909962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee3ae3-c046-43f5-8d93-e762660a8eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:14.911339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee3ae3-c046-43f5-8d93-e762660a8eaa None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:14.911808 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee3ae3-c046-43f5-8d93-e762660a8eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:14.912226 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee3ae3-c046-43f5-8d93-e762660a8eaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:14.924992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee3ae3-c046-43f5-8d93-e762660a8eaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:14.928093 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee3ae3-c046-43f5-8d93-e762660a8eaa None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:14.941425 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:14.950703 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-d0ee3ae3-c046-43f5-8d93-e762660a8eaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:14.950703 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:14.950703 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:14.950703 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:14.950703 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:14.950703 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:14.950703 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:14.950703 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:14.950703 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:14.950703 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:14.950703 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:14.950703 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:14.950703 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:14.950703 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:14.950703 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:14.951672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:14.951672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:14.951672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:14.951672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:14.951672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:14.951672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:14.951672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:14.951672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:14.951672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:14.951672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:14.951672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:14.951672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:14.951672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:14.951672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:14.951672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:14.951672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:14.951672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:14.952666 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:14.952666 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:14.952666 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:14.952666 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:14.954825 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 411/799] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:14.955985 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee3ae3-c046-43f5-8d93-e762660a8eaa None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:15.006146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c9671-06f7-4ccb-bd1b-67489d9ec272 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:15.042342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ddc67-1304-4501-91cc-2ba28a6929e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:15.043161 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ddc67-1304-4501-91cc-2ba28a6929e5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:15.043706 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ddc67-1304-4501-91cc-2ba28a6929e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:15.043943 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ddc67-1304-4501-91cc-2ba28a6929e5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:15.051779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ddc67-1304-4501-91cc-2ba28a6929e5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:15.053335 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ddc67-1304-4501-91cc-2ba28a6929e5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:15.066354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-463ddc67-1304-4501-91cc-2ba28a6929e5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 04:47:15.066354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:15.066354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:15.066354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:15.066354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:15.066354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:15.066354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:15.066354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:15.066354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:15.066354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:15.066354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:15.066354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:15.066354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:47:15.066354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:47:15.066354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:47:15.066354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:47:15.067597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:15.067597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:15.067597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:47:15.067597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:15.067597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:47:15.067597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:15.067597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:15.067597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:15.067597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:15.067597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:15.067597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:15.067597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:15.067597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:15.067597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:15.067597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:15.067597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:15.067597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:15.067597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:15.068790 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:15.068790 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:15.068790 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:47:15.068790 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:47:15.068790 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:47:15.068790 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:47:15.068790 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:47:15.068790 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:47:15.068790 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 04:47:15.068790 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:15.070595 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 389/800] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Jul 26 04:47:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:15.071470 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ddc67-1304-4501-91cc-2ba28a6929e5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:15.073299 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee3ae3-c046-43f5-8d93-e762660a8eaa None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:15.111440 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08326289-bb86-4ede-970f-3a25ec1a4e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:15.112343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08326289-bb86-4ede-970f-3a25ec1a4e95 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:15.112794 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08326289-bb86-4ede-970f-3a25ec1a4e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:15.113298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08326289-bb86-4ede-970f-3a25ec1a4e95 None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:15.114143 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08326289-bb86-4ede-970f-3a25ec1a4e95 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:15.114595 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08326289-bb86-4ede-970f-3a25ec1a4e95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:15.116321 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08326289-bb86-4ede-970f-3a25ec1a4e95 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:15.127074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 412/801] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Jul 26 04:47:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:15.128357 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08326289-bb86-4ede-970f-3a25ec1a4e95 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:15.129641 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:15.177403 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ddc67-1304-4501-91cc-2ba28a6929e5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:15.200429 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:15.214290 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb22e1-08ad-4359-b9c0-6dc0be5e478b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:15.215089 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb22e1-08ad-4359-b9c0-6dc0be5e478b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:15.216532 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb22e1-08ad-4359-b9c0-6dc0be5e478b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:15.216532 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb22e1-08ad-4359-b9c0-6dc0be5e478b None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:15.217089 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb22e1-08ad-4359-b9c0-6dc0be5e478b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:15.217507 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb22e1-08ad-4359-b9c0-6dc0be5e478b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:15.218992 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb22e1-08ad-4359-b9c0-6dc0be5e478b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:15.232753 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 390/802] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:15.238117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb22e1-08ad-4359-b9c0-6dc0be5e478b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:15.252137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08326289-bb86-4ede-970f-3a25ec1a4e95 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:15.290533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa3752-ba76-4ea1-bc05-f7287be4f5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:15.291473 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa3752-ba76-4ea1-bc05-f7287be4f5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:15.291853 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa3752-ba76-4ea1-bc05-f7287be4f5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:15.292131 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa3752-ba76-4ea1-bc05-f7287be4f5e4 None admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:15.292958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa3752-ba76-4ea1-bc05-f7287be4f5e4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:15.293272 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa3752-ba76-4ea1-bc05-f7287be4f5e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:15.294755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa3752-ba76-4ea1-bc05-f7287be4f5e4 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:15.316876 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 413/803] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Jul 26 04:47:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:15.317748 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa3752-ba76-4ea1-bc05-f7287be4f5e4 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:15.354847 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb22e1-08ad-4359-b9c0-6dc0be5e478b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:15.372848 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:15.402083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae266e-cb0c-4c7a-b0e5-fe97b5f1bcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:15.402777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae266e-cb0c-4c7a-b0e5-fe97b5f1bcd8 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:15.403067 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae266e-cb0c-4c7a-b0e5-fe97b5f1bcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:15.403324 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae266e-cb0c-4c7a-b0e5-fe97b5f1bcd8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:15.404080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae266e-cb0c-4c7a-b0e5-fe97b5f1bcd8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:15.406012 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae266e-cb0c-4c7a-b0e5-fe97b5f1bcd8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:15.433494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa3752-ba76-4ea1-bc05-f7287be4f5e4 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:15.434853 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 391/804] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 04:47:15 2022] POST /identity/v3/endpoints => generated 346 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:47:15.436364 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ee1d1-1259-4b47-b60e-14c7bb75fb89 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:15.436796 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ee1d1-1259-4b47-b60e-14c7bb75fb89 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:15.437242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ee1d1-1259-4b47-b60e-14c7bb75fb89 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:15.438775 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 392/805] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:15.439763 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ee1d1-1259-4b47-b60e-14c7bb75fb89 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:15.472101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:15.492886 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b89af1f-78da-4491-a743-562356cfc0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:15.493999 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89af1f-78da-4491-a743-562356cfc0ec None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:15.494604 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89af1f-78da-4491-a743-562356cfc0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:15.495083 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89af1f-78da-4491-a743-562356cfc0ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:15.503760 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b89af1f-78da-4491-a743-562356cfc0ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:15.506490 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b89af1f-78da-4491-a743-562356cfc0ec None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:15.525033 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-3b89af1f-78da-4491-a743-562356cfc0ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:15.525033 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:15.525033 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:15.525033 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:15.525033 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:15.525033 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:15.525033 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:15.525033 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:15.525033 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:15.525033 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:15.525033 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:15.525033 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:15.525033 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:47:15.525033 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:47:15.525033 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:47:15.526207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:47:15.526207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:15.526207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:15.526207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:15.526207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:15.526207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:15.526207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:15.526207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:15.526207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:15.526207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:15.526207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:15.526207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:15.526207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:15.526207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:15.526207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:15.526207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:15.526207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:15.527102 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:47:15.527102 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:47:15.527102 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:47:15.527102 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:47:15.527102 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:47:15.527102 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:47:15.527102 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:15.527102 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:15.535316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 414/806] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:47:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:15.535316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f0e7a-7009-4b4a-b74c-ce302bfce711 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:15.535316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f0e7a-7009-4b4a-b74c-ce302bfce711 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:15.535316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f0e7a-7009-4b4a-b74c-ce302bfce711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:15.569774 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-e69f0e7a-7009-4b4a-b74c-ce302bfce711 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:15.570691 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ee1d1-1259-4b47-b60e-14c7bb75fb89 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:15.571743 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-e69f0e7a-7009-4b4a-b74c-ce302bfce711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:15.612680 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a8151-53cb-41b2-8536-0bc593d8230e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:15.613336 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a8151-53cb-41b2-8536-0bc593d8230e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:15.613622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a8151-53cb-41b2-8536-0bc593d8230e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:15.613899 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a8151-53cb-41b2-8536-0bc593d8230e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:15.664905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:15.681745 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 415/807] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:15.682757 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f0e7a-7009-4b4a-b74c-ce302bfce711 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:15.704491 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a8151-53cb-41b2-8536-0bc593d8230e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:15.706436 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a8151-53cb-41b2-8536-0bc593d8230e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:15.795705 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f0e7a-7009-4b4a-b74c-ce302bfce711 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:15.807186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:15.829180 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 393/808] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:47:15 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 04:47:15.829180 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16218898-bc93-4920-a65a-2e1f8225f4a1 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:15.829180 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16218898-bc93-4920-a65a-2e1f8225f4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:15.829180 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16218898-bc93-4920-a65a-2e1f8225f4a1 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:15.829180 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 394/809] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:15.829180 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16218898-bc93-4920-a65a-2e1f8225f4a1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:15.843424 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f2a0c-20c5-4919-8d00-832dbd50ae4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:15.844262 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f2a0c-20c5-4919-8d00-832dbd50ae4e None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:15.844419 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f2a0c-20c5-4919-8d00-832dbd50ae4e None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:15.844679 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f2a0c-20c5-4919-8d00-832dbd50ae4e None admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:15.845501 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f2a0c-20c5-4919-8d00-832dbd50ae4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:15.845814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f2a0c-20c5-4919-8d00-832dbd50ae4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:15.847320 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f2a0c-20c5-4919-8d00-832dbd50ae4e None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:15.861242 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 416/810] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:47:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:15.863484 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c7d0f-90c4-4115-af0d-3d4a876cb63c None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:15.864424 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c7d0f-90c4-4115-af0d-3d4a876cb63c None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:15.865053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c7d0f-90c4-4115-af0d-3d4a876cb63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:15.884490 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-323c7d0f-90c4-4115-af0d-3d4a876cb63c None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:15.885745 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-323c7d0f-90c4-4115-af0d-3d4a876cb63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:15.913638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:15.948128 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16218898-bc93-4920-a65a-2e1f8225f4a1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:15.959594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 417/811] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:15.959594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c7d0f-90c4-4115-af0d-3d4a876cb63c None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:15.999129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713fa61-bb2d-4d1d-84d1-393556c84ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:15.999929 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713fa61-bb2d-4d1d-84d1-393556c84ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:16.000279 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713fa61-bb2d-4d1d-84d1-393556c84ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:16.000605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713fa61-bb2d-4d1d-84d1-393556c84ec8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:16.012798 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713fa61-bb2d-4d1d-84d1-393556c84ec8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:47:16.017802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b713fa61-bb2d-4d1d-84d1-393556c84ec8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:47:16.017802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:16.017802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:16.017802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:16.017802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:16.017802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:16.017802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:16.017802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:16.017802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:16.017802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:16.017802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:16.017802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:16.017802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:47:16.017802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:47:16.017802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:47:16.017802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:47:16.019056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:47:16.019056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:47:16.019056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:47:16.019056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:47:16.019056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:16.019056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:16.019056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:16.019056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:16.019056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:16.019056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:16.019056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:16.019056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:16.019056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:16.019056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:16.019056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:16.019056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:16.020246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:16.020246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:16.020246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:16.020246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:16.020246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:47:16.020246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:47:16.020246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:47:16.020246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:47:16.020246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:47:16.020246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:47:16.020246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:47:16.020246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:16.023971 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 395/812] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jul 26 04:47:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:16.024840 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713fa61-bb2d-4d1d-84d1-393556c84ec8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:16.074347 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:16.077065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c7d0f-90c4-4115-af0d-3d4a876cb63c None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:16.115704 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3dbd1-cfd7-4a7d-b582-fd30209f8fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:16.116362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3dbd1-cfd7-4a7d-b582-fd30209f8fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:16.116642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3dbd1-cfd7-4a7d-b582-fd30209f8fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:16.116925 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3dbd1-cfd7-4a7d-b582-fd30209f8fc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:16.127468 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c3dbd1-cfd7-4a7d-b582-fd30209f8fc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:16.128960 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c3dbd1-cfd7-4a7d-b582-fd30209f8fc8 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:16.129171 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713fa61-bb2d-4d1d-84d1-393556c84ec8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:16.146095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-40c3dbd1-cfd7-4a7d-b582-fd30209f8fc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:16.146095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:16.146095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:16.146095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:16.146095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:16.146095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:16.146095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:16.146095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:16.146095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:16.146095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:16.146095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:16.146095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:16.146095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:16.146095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:16.146095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:16.147057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:16.147057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:16.147057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:16.147057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:16.147057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:16.147057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:16.147057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:16.147057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:16.147057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:16.147057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:16.147057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:16.147057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:16.147057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:16.147057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:16.147057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:16.147057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:16.147057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:16.147982 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:16.147982 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:16.147982 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:16.147982 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:16.147982 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 418/813] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:16.148262 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3dbd1-cfd7-4a7d-b582-fd30209f8fc8 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:16.165512 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:16.172546 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09ff4f-703b-4cf1-b858-ba22d79f6725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:16.173235 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09ff4f-703b-4cf1-b858-ba22d79f6725 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:16.173514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09ff4f-703b-4cf1-b858-ba22d79f6725 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:16.173790 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09ff4f-703b-4cf1-b858-ba22d79f6725 None admin] PATH_INFO: `/v3/projects/d583e0be8ef74766bffd6c3c07af18f1/users/d5218319aaa5424882714bffcbeee730/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:16.175993 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a09ff4f-703b-4cf1-b858-ba22d79f6725 None admin] RBAC: Authorizing `identity:create_grant(project_id=d583e0be8ef74766bffd6c3c07af18f1, user_id=d5218319aaa5424882714bffcbeee730, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:16.177231 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a09ff4f-703b-4cf1-b858-ba22d79f6725 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:16.191867 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 396/814] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jul 26 04:47:16 2022] PUT /identity/v3/projects/d583e0be8ef74766bffd6c3c07af18f1/users/d5218319aaa5424882714bffcbeee730/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:47:16.192741 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09ff4f-703b-4cf1-b858-ba22d79f6725 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:16.295959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3dbd1-cfd7-4a7d-b582-fd30209f8fc8 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:16.311838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09ff4f-703b-4cf1-b858-ba22d79f6725 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:16.344209 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5231c696-f444-43f5-bbcb-70b786b8a994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:16.345545 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231c696-f444-43f5-bbcb-70b786b8a994 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:16.346044 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231c696-f444-43f5-bbcb-70b786b8a994 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:16.346672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231c696-f444-43f5-bbcb-70b786b8a994 None admin] PATH_INFO: `/v3/services/network` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:16.347919 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5231c696-f444-43f5-bbcb-70b786b8a994 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:16.349960 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5231c696-f444-43f5-bbcb-70b786b8a994 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:16.355770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d13e6-6f7a-4fe5-a432-024b259fd43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:16.356333 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d13e6-6f7a-4fe5-a432-024b259fd43d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:16.356590 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d13e6-6f7a-4fe5-a432-024b259fd43d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:16.356772 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d13e6-6f7a-4fe5-a432-024b259fd43d None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:16.357514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d13e6-6f7a-4fe5-a432-024b259fd43d None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:16.357825 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d13e6-6f7a-4fe5-a432-024b259fd43d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:16.359312 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d13e6-6f7a-4fe5-a432-024b259fd43d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:16.369680 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 397/815] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jul 26 04:47:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:16.370602 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d13e6-6f7a-4fe5-a432-024b259fd43d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:16.371096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-5231c696-f444-43f5-bbcb-70b786b8a994 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 04:47:16.371096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:16.371096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 04:47:16.371096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 04:47:16.371096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 04:47:16.371096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 04:47:16.371096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 04:47:16.371096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:16.371096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 04:47:16.371096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:16.371096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:47:16.371096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:16.371096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:16.371096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:16.371096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:16.371096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:16.371096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:16.373134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:16.373134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:16.373134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:16.373134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:16.373134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:16.373134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:16.373134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 04:47:16.373134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 04:47:16.373134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 04:47:16.373134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 04:47:16.373134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:16.373134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:16.373134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:16.373134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:16.373134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:16.373134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:16.375656 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:16.375656 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:16.375656 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:16.375656 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:16.375656 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:16.375656 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:16.375656 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:16.375656 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:16.375656 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:16.375656 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:16.375656 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 04:47:16.375656 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:16.375656 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 04:47:16.375656 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:16.382130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 419/816] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:47:16 2022] GET /identity/v3/services/network => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:16.384647 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5231c696-f444-43f5-bbcb-70b786b8a994 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:16.408622 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:16.472494 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:16.518636 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5231c696-f444-43f5-bbcb-70b786b8a994 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:16.535295 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d13e6-6f7a-4fe5-a432-024b259fd43d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:16.569777 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d17c0-5c6b-453d-89d5-23270e4bdd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:16.571131 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d17c0-5c6b-453d-89d5-23270e4bdd72 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:16.571977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d17c0-5c6b-453d-89d5-23270e4bdd72 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:16.572771 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d17c0-5c6b-453d-89d5-23270e4bdd72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:16.579276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89d338-f7f6-484e-8a9d-78cfa8480ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:16.579972 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89d338-f7f6-484e-8a9d-78cfa8480ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:16.580251 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89d338-f7f6-484e-8a9d-78cfa8480ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:16.580523 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89d338-f7f6-484e-8a9d-78cfa8480ef3 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:16.581221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89d338-f7f6-484e-8a9d-78cfa8480ef3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:16.581523 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89d338-f7f6-484e-8a9d-78cfa8480ef3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:16.582894 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89d338-f7f6-484e-8a9d-78cfa8480ef3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:16.587372 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d17c0-5c6b-453d-89d5-23270e4bdd72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:16.589387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d17c0-5c6b-453d-89d5-23270e4bdd72 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:16.599675 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 398/817] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:16.599675 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89d338-f7f6-484e-8a9d-78cfa8480ef3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:16.629228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-0e8d17c0-5c6b-453d-89d5-23270e4bdd72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:16.629228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:16.629228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:16.629228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:16.629228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:16.629228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:16.629228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:16.629228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:16.629228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:16.629228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:16.629228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:16.629228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:16.629228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:16.629228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:16.629228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:16.630219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:16.630219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:16.630219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:16.630219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:16.630219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:16.630219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:16.630219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:16.630219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:16.630219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:16.630219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:16.630219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:16.630219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:16.630219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:16.630219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:16.630219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:16.630219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:16.630219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:16.631113 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:16.631113 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:16.631113 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:16.631113 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:16.631113 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 420/818] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:16.631113 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d17c0-5c6b-453d-89d5-23270e4bdd72 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:16.703028 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:16.708516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89d338-f7f6-484e-8a9d-78cfa8480ef3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:16.724044 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:16.744211 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d17c0-5c6b-453d-89d5-23270e4bdd72 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:16.758102 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27bd68-48db-4d62-aa04-c9e57f8cbc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:16.759104 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27bd68-48db-4d62-aa04-c9e57f8cbc5e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:16.759388 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27bd68-48db-4d62-aa04-c9e57f8cbc5e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:16.759684 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27bd68-48db-4d62-aa04-c9e57f8cbc5e None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:16.760736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba27bd68-48db-4d62-aa04-c9e57f8cbc5e None admin] RBAC: Adding query filter params (name=network) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:16.761044 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba27bd68-48db-4d62-aa04-c9e57f8cbc5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:16.762584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba27bd68-48db-4d62-aa04-c9e57f8cbc5e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:16.771583 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 399/819] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Jul 26 04:47:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:16.771583 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27bd68-48db-4d62-aa04-c9e57f8cbc5e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:16.785653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64166cad-5604-4b1c-8ffc-aa3eb9748d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:16.786693 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64166cad-5604-4b1c-8ffc-aa3eb9748d21 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:16.787152 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64166cad-5604-4b1c-8ffc-aa3eb9748d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:16.787686 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64166cad-5604-4b1c-8ffc-aa3eb9748d21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:16.795330 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64166cad-5604-4b1c-8ffc-aa3eb9748d21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:16.797247 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64166cad-5604-4b1c-8ffc-aa3eb9748d21 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:16.809909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-64166cad-5604-4b1c-8ffc-aa3eb9748d21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:16.809909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:16.809909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:16.809909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:16.809909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:16.809909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:16.809909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:16.809909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:16.809909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:16.809909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:16.809909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:16.809909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:16.809909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:47:16.809909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:47:16.809909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:47:16.810857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:47:16.810857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:16.810857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:16.810857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:16.810857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:16.810857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:16.810857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:16.810857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:16.810857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:16.810857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:16.810857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:16.810857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:16.810857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:16.810857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:16.810857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:16.810857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:16.810857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:16.811773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:47:16.811773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:47:16.811773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:47:16.811773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:47:16.811773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:47:16.811773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:47:16.811773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:16.811773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:16.814319 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 421/820] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:47:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:16.815452 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64166cad-5604-4b1c-8ffc-aa3eb9748d21 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:16.891412 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27bd68-48db-4d62-aa04-c9e57f8cbc5e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:16.916112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64166cad-5604-4b1c-8ffc-aa3eb9748d21 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:16.935282 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548448c-fcbf-47f5-be98-d3de81fce993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:16.935957 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548448c-fcbf-47f5-be98-d3de81fce993 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:16.936230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548448c-fcbf-47f5-be98-d3de81fce993 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:16.936502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548448c-fcbf-47f5-be98-d3de81fce993 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:16.937192 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2548448c-fcbf-47f5-be98-d3de81fce993 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:16.937497 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2548448c-fcbf-47f5-be98-d3de81fce993 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:16.939313 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2548448c-fcbf-47f5-be98-d3de81fce993 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:16.948022 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:16.957996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d7c00-8ca0-4006-adf2-a86e406349a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:16.957996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d7c00-8ca0-4006-adf2-a86e406349a7 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:16.957996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d7c00-8ca0-4006-adf2-a86e406349a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:16.957996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d7c00-8ca0-4006-adf2-a86e406349a7 None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:16.958383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d7c00-8ca0-4006-adf2-a86e406349a7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:16.958383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d7c00-8ca0-4006-adf2-a86e406349a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:16.958383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d7c00-8ca0-4006-adf2-a86e406349a7 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:16.959052 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 400/821] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:16.960209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548448c-fcbf-47f5-be98-d3de81fce993 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:16.967607 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 422/822] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Jul 26 04:47:16 2022] GET /identity/v3/services?type=network => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:16.967607 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d7c00-8ca0-4006-adf2-a86e406349a7 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:17.004979 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:17.078578 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548448c-fcbf-47f5-be98-d3de81fce993 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:17.092111 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d7c00-8ca0-4006-adf2-a86e406349a7 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:17.125998 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aebeb9-540d-4f01-968a-889652e2903a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:17.126640 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aebeb9-540d-4f01-968a-889652e2903a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:17.126920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aebeb9-540d-4f01-968a-889652e2903a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:17.127193 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aebeb9-540d-4f01-968a-889652e2903a None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:17.127976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aebeb9-540d-4f01-968a-889652e2903a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:17.128283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aebeb9-540d-4f01-968a-889652e2903a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:17.129734 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aebeb9-540d-4f01-968a-889652e2903a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:17.136302 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f8aa6-b3d2-4d8b-b2f9-cfcda8122d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:17.137309 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f8aa6-b3d2-4d8b-b2f9-cfcda8122d6e None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:17.137806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f8aa6-b3d2-4d8b-b2f9-cfcda8122d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:17.138269 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f8aa6-b3d2-4d8b-b2f9-cfcda8122d6e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:17.145777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 401/823] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:47:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:17.146652 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aebeb9-540d-4f01-968a-889652e2903a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:17.149436 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f8aa6-b3d2-4d8b-b2f9-cfcda8122d6e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:17.152407 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f8aa6-b3d2-4d8b-b2f9-cfcda8122d6e None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:17.167958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-871f8aa6-b3d2-4d8b-b2f9-cfcda8122d6e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 04:47:17.167958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:17.167958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:17.167958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:17.167958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:17.167958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:17.167958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:17.167958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:17.167958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:17.167958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:17.167958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:17.167958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:17.167958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:47:17.167958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:47:17.167958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:47:17.167958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:47:17.168939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:17.168939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:17.168939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:47:17.168939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:17.168939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:47:17.168939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:17.168939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:17.168939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:17.168939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:17.168939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:17.168939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:17.168939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:17.168939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:17.168939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:17.168939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:17.168939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:17.168939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:17.168939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:17.169881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:17.169881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:17.169881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:47:17.169881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:47:17.169881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:47:17.169881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:47:17.169881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:47:17.169881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:47:17.169881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 04:47:17.169881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:17.173100 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 423/824] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Jul 26 04:47:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:17.174103 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f8aa6-b3d2-4d8b-b2f9-cfcda8122d6e None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:17.254093 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:17.256095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:17.260516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aebeb9-540d-4f01-968a-889652e2903a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:17.298984 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ed99a-5338-4026-8f21-6524fb080616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:17.299926 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ed99a-5338-4026-8f21-6524fb080616 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:17.300340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ed99a-5338-4026-8f21-6524fb080616 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:17.300698 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ed99a-5338-4026-8f21-6524fb080616 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:17.301576 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ed99a-5338-4026-8f21-6524fb080616 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:17.303052 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ed99a-5338-4026-8f21-6524fb080616 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:17.311115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f8aa6-b3d2-4d8b-b2f9-cfcda8122d6e None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:17.336510 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 402/825] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 04:47:17 2022] POST /identity/v3/endpoints => generated 327 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:47:17.337879 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ed99a-5338-4026-8f21-6524fb080616 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:17.357228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5637f88-289d-4c91-84bf-386ced5354ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:17.358109 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5637f88-289d-4c91-84bf-386ced5354ff None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:17.358528 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5637f88-289d-4c91-84bf-386ced5354ff None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:17.358957 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5637f88-289d-4c91-84bf-386ced5354ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:17.359971 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5637f88-289d-4c91-84bf-386ced5354ff None admin] RBAC: Adding query filter params (role.id=72bf8bd23d2c4b76ad32b7b35d6db412, scope.project.id=d583e0be8ef74766bffd6c3c07af18f1, user.id=2faeed9002ee4154bc980c146cd06bcc) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:17.360434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5637f88-289d-4c91-84bf-386ced5354ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:17.362063 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5637f88-289d-4c91-84bf-386ced5354ff None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:17.369980 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 424/826] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:47:17 2022] GET /identity/v3/role_assignments?user.id=2faeed9002ee4154bc980c146cd06bcc&scope.project.id=d583e0be8ef74766bffd6c3c07af18f1&role.id=72bf8bd23d2c4b76ad32b7b35d6db412 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:17.522452 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ed99a-5338-4026-8f21-6524fb080616 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:17.562147 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea3a36-7cb2-4789-8a87-7649fdfc0f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:17.562808 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea3a36-7cb2-4789-8a87-7649fdfc0f8d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:17.563091 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea3a36-7cb2-4789-8a87-7649fdfc0f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:17.563369 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea3a36-7cb2-4789-8a87-7649fdfc0f8d None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:17.564242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea3a36-7cb2-4789-8a87-7649fdfc0f8d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:17.564550 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea3a36-7cb2-4789-8a87-7649fdfc0f8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:17.566063 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea3a36-7cb2-4789-8a87-7649fdfc0f8d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:17.587533 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 403/827] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Jul 26 04:47:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:17.594911 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5637f88-289d-4c91-84bf-386ced5354ff None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:17.663403 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81653b-f054-405a-ab1c-97b8fe38ef65 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:17.663790 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81653b-f054-405a-ab1c-97b8fe38ef65 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:17.663997 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81653b-f054-405a-ab1c-97b8fe38ef65 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:17.665795 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 404/828] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:17.671230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d70308-8930-4da3-8727-3ba76708a3d7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:17.671515 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d70308-8930-4da3-8727-3ba76708a3d7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:17.671919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d70308-8930-4da3-8727-3ba76708a3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:17.696979 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f5d70308-8930-4da3-8727-3ba76708a3d7 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:17.698301 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f5d70308-8930-4da3-8727-3ba76708a3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:17.701506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:17.718603 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5637f88-289d-4c91-84bf-386ced5354ff None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:17.732021 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:17.760811 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d2873-8d4b-4334-b2b7-ebcaebdafa46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:17.761936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d2873-8d4b-4334-b2b7-ebcaebdafa46 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:17.762449 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d2873-8d4b-4334-b2b7-ebcaebdafa46 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:17.763009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d2873-8d4b-4334-b2b7-ebcaebdafa46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:17.775512 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d2873-8d4b-4334-b2b7-ebcaebdafa46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:17.777151 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d2873-8d4b-4334-b2b7-ebcaebdafa46 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:17.781860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 405/829] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:17.796930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-6d6d2873-8d4b-4334-b2b7-ebcaebdafa46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:17.796930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:17.796930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:17.796930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:17.796930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:17.796930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:17.796930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:17.796930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:17.796930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:17.796930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:17.796930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:17.796930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:17.796930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:17.796930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:17.796930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:17.797900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:17.797900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:17.797900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:17.797900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:17.797900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:17.797900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:17.797900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:17.797900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:17.797900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:17.797900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:17.797900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:17.797900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:17.797900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:17.797900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:17.797900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:17.797900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:17.797900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:17.798797 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:17.798797 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:17.798797 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:17.798797 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:17.798797 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 425/830] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:17.803921 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d70308-8930-4da3-8727-3ba76708a3d7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:17.814813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d2873-8d4b-4334-b2b7-ebcaebdafa46 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:17.915482 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d70308-8930-4da3-8727-3ba76708a3d7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:17.952748 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c810b-50bd-4d69-9c3f-c18c14383f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:17.953343 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c810b-50bd-4d69-9c3f-c18c14383f0e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:17.953627 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c810b-50bd-4d69-9c3f-c18c14383f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:17.953906 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c810b-50bd-4d69-9c3f-c18c14383f0e None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:17.954649 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c810b-50bd-4d69-9c3f-c18c14383f0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:17.954955 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c810b-50bd-4d69-9c3f-c18c14383f0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:17.956410 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c810b-50bd-4d69-9c3f-c18c14383f0e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:17.964595 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d2873-8d4b-4334-b2b7-ebcaebdafa46 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:17.966975 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:17.975899 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 406/831] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:17.977590 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c810b-50bd-4d69-9c3f-c18c14383f0e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:18.011356 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c726f-3dc4-4b02-a693-a659e4de5737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:18.012184 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c726f-3dc4-4b02-a693-a659e4de5737 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:18.012493 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c726f-3dc4-4b02-a693-a659e4de5737 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:18.012768 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c726f-3dc4-4b02-a693-a659e4de5737 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:18.013475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c726f-3dc4-4b02-a693-a659e4de5737 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:18.014984 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c726f-3dc4-4b02-a693-a659e4de5737 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:18.024747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-192c726f-3dc4-4b02-a693-a659e4de5737 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 04:47:18.024747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:18.024747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 04:47:18.024747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 04:47:18.024747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 04:47:18.024747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 04:47:18.024747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 04:47:18.024747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:18.024747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 04:47:18.024747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:18.024747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:47:18.024747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:18.024747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:18.024747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:18.024747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:18.024747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:18.024747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:18.025972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:18.025972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:18.025972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:18.025972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:18.025972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:18.025972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:18.025972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 04:47:18.025972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 04:47:18.025972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 04:47:18.025972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 04:47:18.025972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:18.025972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:18.025972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:18.025972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:18.025972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:18.025972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:18.026811 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:18.026811 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:18.026811 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:18.026811 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:18.026811 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:18.026811 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:18.026811 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:18.026811 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:18.026811 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:18.026811 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:18.026811 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 04:47:18.026811 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:18.026811 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 04:47:18.026811 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:18.031133 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 426/832] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jul 26 04:47:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:18.032705 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c726f-3dc4-4b02-a693-a659e4de5737 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:18.061361 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:18.121998 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c810b-50bd-4d69-9c3f-c18c14383f0e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:18.144628 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c726f-3dc4-4b02-a693-a659e4de5737 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:18.161650 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56c6d7-5e1a-447a-a9d3-41a4e6b83fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:18.162317 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56c6d7-5e1a-447a-a9d3-41a4e6b83fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:18.162600 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56c6d7-5e1a-447a-a9d3-41a4e6b83fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:18.162874 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56c6d7-5e1a-447a-a9d3-41a4e6b83fb8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:18.170408 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd56c6d7-5e1a-447a-a9d3-41a4e6b83fb8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:18.171894 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd56c6d7-5e1a-447a-a9d3-41a4e6b83fb8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:18.184302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-fd56c6d7-5e1a-447a-a9d3-41a4e6b83fb8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:18.184302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:18.184302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:18.184302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:18.184302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:18.184302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:18.184302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:18.184302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:18.184302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:18.184302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:18.184302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:18.184302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:18.184302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:47:18.184302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:47:18.184302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:47:18.185242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:47:18.185242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:18.185242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:18.185242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:18.185242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:18.185242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:18.185242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:18.185242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:18.185242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:18.185242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:18.185242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:18.185242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:18.185242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:18.185242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:18.185242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:18.185242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:18.185242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:18.186235 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:47:18.186235 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:47:18.186235 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:47:18.186235 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:47:18.186235 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:47:18.186235 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:47:18.186235 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:18.186235 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:18.190260 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 407/833] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:47:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:18.191190 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56c6d7-5e1a-447a-a9d3-41a4e6b83fb8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:18.192784 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4eacbb-93c7-47ab-a769-ff2985c3471b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:18.193684 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4eacbb-93c7-47ab-a769-ff2985c3471b None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:18.194508 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4eacbb-93c7-47ab-a769-ff2985c3471b None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:18.195819 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4eacbb-93c7-47ab-a769-ff2985c3471b None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:18.196679 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4eacbb-93c7-47ab-a769-ff2985c3471b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:18.196993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4eacbb-93c7-47ab-a769-ff2985c3471b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:18.198463 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4eacbb-93c7-47ab-a769-ff2985c3471b None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:18.208561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 427/834] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jul 26 04:47:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:18.209487 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4eacbb-93c7-47ab-a769-ff2985c3471b None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:18.265176 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:18.312271 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56c6d7-5e1a-447a-a9d3-41a4e6b83fb8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:18.323759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:18.339357 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4eacbb-93c7-47ab-a769-ff2985c3471b None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:18.360817 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0cce43-1ec7-4ac2-9258-abefbaa7df34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:18.361556 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0cce43-1ec7-4ac2-9258-abefbaa7df34 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:18.361841 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0cce43-1ec7-4ac2-9258-abefbaa7df34 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:18.362115 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0cce43-1ec7-4ac2-9258-abefbaa7df34 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:18.362877 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0cce43-1ec7-4ac2-9258-abefbaa7df34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:18.363182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0cce43-1ec7-4ac2-9258-abefbaa7df34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:18.364915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0cce43-1ec7-4ac2-9258-abefbaa7df34 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:18.375024 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709432d0-ba88-4f15-a32b-21daf1db19ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:18.375662 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709432d0-ba88-4f15-a32b-21daf1db19ff None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:18.375950 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709432d0-ba88-4f15-a32b-21daf1db19ff None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:18.376225 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709432d0-ba88-4f15-a32b-21daf1db19ff None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:18.376909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709432d0-ba88-4f15-a32b-21daf1db19ff None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:18.377215 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709432d0-ba88-4f15-a32b-21daf1db19ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:18.378627 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709432d0-ba88-4f15-a32b-21daf1db19ff None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:18.387583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 428/835] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jul 26 04:47:18 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:18.387583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cfd3f1-6f88-4ed7-bcb8-4306cf9c4da4 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:18.387583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cfd3f1-6f88-4ed7-bcb8-4306cf9c4da4 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:18.387583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cfd3f1-6f88-4ed7-bcb8-4306cf9c4da4 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:18.387989 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 429/836] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:18.388260 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 408/837] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:47:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:18.390286 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cfd3f1-6f88-4ed7-bcb8-4306cf9c4da4 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:18.393166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee4d9c-b35d-44ca-9bf9-429c0d62f70b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:18.394187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee4d9c-b35d-44ca-9bf9-429c0d62f70b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:18.394469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee4d9c-b35d-44ca-9bf9-429c0d62f70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:18.420511 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f2ee4d9c-b35d-44ca-9bf9-429c0d62f70b None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:18.422806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f2ee4d9c-b35d-44ca-9bf9-429c0d62f70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:18.514424 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cfd3f1-6f88-4ed7-bcb8-4306cf9c4da4 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:18.523591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 409/838] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 04:47:18.545452 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee4d9c-b35d-44ca-9bf9-429c0d62f70b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:18.566608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1ec86-6ac0-45b9-ae33-7b75e1298056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:18.567636 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1ec86-6ac0-45b9-ae33-7b75e1298056 None admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:18.567636 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1ec86-6ac0-45b9-ae33-7b75e1298056 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:18.567807 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1ec86-6ac0-45b9-ae33-7b75e1298056 None admin] PATH_INFO: `/v3/projects/d583e0be8ef74766bffd6c3c07af18f1/users/3a8f3eb811ae40ad9cb64261f95a992e/roles/e5048f4e7c6b4494a15d628758090380` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:18.587848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b1ec86-6ac0-45b9-ae33-7b75e1298056 None admin] RBAC: Authorizing `identity:create_grant(project_id=d583e0be8ef74766bffd6c3c07af18f1, user_id=3a8f3eb811ae40ad9cb64261f95a992e, role_id=e5048f4e7c6b4494a15d628758090380)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:18.589410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b1ec86-6ac0-45b9-ae33-7b75e1298056 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:18.606515 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 430/839] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jul 26 04:47:18 2022] PUT /identity/v3/projects/d583e0be8ef74766bffd6c3c07af18f1/users/3a8f3eb811ae40ad9cb64261f95a992e/roles/e5048f4e7c6b4494a15d628758090380 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 04:47:18.610051 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:18.672053 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee4d9c-b35d-44ca-9bf9-429c0d62f70b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:18.712118 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d47125-cb7c-4b12-a463-6d54b7a9067d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:18.712804 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d47125-cb7c-4b12-a463-6d54b7a9067d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:18.713126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d47125-cb7c-4b12-a463-6d54b7a9067d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:18.713435 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d47125-cb7c-4b12-a463-6d54b7a9067d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:18.726340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d47125-cb7c-4b12-a463-6d54b7a9067d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:47:18.731401 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-f2d47125-cb7c-4b12-a463-6d54b7a9067d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:47:18.731401 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:18.731401 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:18.731401 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:18.731401 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:18.731401 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:18.731401 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:18.731401 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:18.731401 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:18.731401 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:18.731401 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:18.731401 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:18.731401 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:47:18.731401 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:47:18.731401 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:47:18.731401 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:47:18.732428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:47:18.732428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:47:18.732428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:47:18.732428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:47:18.732428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:18.732428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:18.732428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:18.732428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:18.732428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:18.732428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:18.732428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:18.732428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:18.732428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:18.732428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:18.732428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:18.732428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:18.733272 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:18.733272 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:18.733272 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:18.733272 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:18.733272 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:47:18.733272 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:47:18.733272 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:47:18.733272 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:47:18.733272 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:47:18.733272 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:47:18.733272 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:47:18.733272 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:18.733272 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 410/840] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:47:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:47:18.739667 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1ec86-6ac0-45b9-ae33-7b75e1298056 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:18.746771 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:18.853291 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1ec86-6ac0-45b9-ae33-7b75e1298056 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:18.889202 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a91a12-8811-419c-b71f-1499c003d1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:18.889841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a91a12-8811-419c-b71f-1499c003d1de None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:18.890120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a91a12-8811-419c-b71f-1499c003d1de None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:18.890396 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a91a12-8811-419c-b71f-1499c003d1de None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:18.891175 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a91a12-8811-419c-b71f-1499c003d1de None admin] RBAC: Adding query filter params (name=admin) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:18.891484 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a91a12-8811-419c-b71f-1499c003d1de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:18.892900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a91a12-8811-419c-b71f-1499c003d1de None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:18.902596 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 431/841] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:47:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:18.908288 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d47125-cb7c-4b12-a463-6d54b7a9067d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:19.016443 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d47125-cb7c-4b12-a463-6d54b7a9067d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:19.028635 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:19.054967 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5058164-d397-45ca-9347-00be3dc8f10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:19.056033 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5058164-d397-45ca-9347-00be3dc8f10d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:19.056525 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5058164-d397-45ca-9347-00be3dc8f10d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:19.057032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5058164-d397-45ca-9347-00be3dc8f10d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:19.068126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5058164-d397-45ca-9347-00be3dc8f10d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:19.069903 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5058164-d397-45ca-9347-00be3dc8f10d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:19.085779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a5058164-d397-45ca-9347-00be3dc8f10d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:19.085779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:19.085779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:19.085779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:19.085779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:19.085779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:19.085779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:19.085779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:19.085779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:19.085779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:19.085779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:19.085779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:19.085779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:19.085779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:19.085779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:19.086725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:19.086725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:19.086725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:19.086725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:19.086725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:19.086725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:19.086725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:19.086725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:19.086725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:19.086725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:19.086725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:19.086725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:19.086725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:19.086725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:19.086725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:19.086725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:19.086725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:19.087663 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:19.087663 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:19.087663 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:19.087663 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:19.089927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 411/842] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:19.096349 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a91a12-8811-419c-b71f-1499c003d1de None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:19.173019 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:19.207091 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a91a12-8811-419c-b71f-1499c003d1de None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:19.242118 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a2072-5677-4e34-be74-fb49b74db7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:19.242753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a2072-5677-4e34-be74-fb49b74db7da None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:19.243040 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a2072-5677-4e34-be74-fb49b74db7da None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:19.243318 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a2072-5677-4e34-be74-fb49b74db7da None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:19.244060 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a2072-5677-4e34-be74-fb49b74db7da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:19.244432 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a2072-5677-4e34-be74-fb49b74db7da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:19.245836 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a2072-5677-4e34-be74-fb49b74db7da None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:19.258612 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 432/843] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:19.264214 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5058164-d397-45ca-9347-00be3dc8f10d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:19.374206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5058164-d397-45ca-9347-00be3dc8f10d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:19.410855 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad25e1f-2ce5-460b-9ab7-6e7d89155126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:19.411525 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad25e1f-2ce5-460b-9ab7-6e7d89155126 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:19.411834 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad25e1f-2ce5-460b-9ab7-6e7d89155126 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:19.412117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad25e1f-2ce5-460b-9ab7-6e7d89155126 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:19.420125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad25e1f-2ce5-460b-9ab7-6e7d89155126 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:19.421591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad25e1f-2ce5-460b-9ab7-6e7d89155126 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:19.433518 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-4ad25e1f-2ce5-460b-9ab7-6e7d89155126 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 04:47:19.433518 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:19.433518 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:19.433518 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:19.433518 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:19.433518 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:19.433518 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:19.433518 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:19.433518 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:19.433518 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:19.433518 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:19.433518 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:19.433518 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:47:19.433518 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:47:19.433518 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:47:19.433518 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:47:19.434490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:19.434490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:19.434490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:47:19.434490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:19.434490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:47:19.434490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:19.434490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:19.434490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:19.434490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:19.434490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:19.434490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:19.434490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:19.434490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:19.434490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:19.434490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:19.434490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:19.434490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:19.434490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:19.435425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:19.435425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:19.435425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:47:19.435425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:47:19.435425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:47:19.435425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:47:19.435425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:47:19.435425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:47:19.435425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 04:47:19.435425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:19.435425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 412/844] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:47:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:19.440685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a2072-5677-4e34-be74-fb49b74db7da None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:19.546948 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a2072-5677-4e34-be74-fb49b74db7da None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:19.574372 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:19.591411 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb971f6-c260-47e5-81f4-1cb28cc12dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:19.592132 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb971f6-c260-47e5-81f4-1cb28cc12dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:19.592443 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb971f6-c260-47e5-81f4-1cb28cc12dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:19.592727 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb971f6-c260-47e5-81f4-1cb28cc12dd3 None admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:19.593694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb971f6-c260-47e5-81f4-1cb28cc12dd3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:19.594006 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb971f6-c260-47e5-81f4-1cb28cc12dd3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:19.595510 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb971f6-c260-47e5-81f4-1cb28cc12dd3 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:19.615919 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 433/845] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Jul 26 04:47:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:19.621349 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad25e1f-2ce5-460b-9ab7-6e7d89155126 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:19.648887 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:19.690746 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63faf20-5603-476b-b252-1ce50458c2f0 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:19.690947 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63faf20-5603-476b-b252-1ce50458c2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:19.691191 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63faf20-5603-476b-b252-1ce50458c2f0 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:19.693102 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 434/846] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:47:19.698952 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23163b4b-92dc-4c14-a022-5d90b95879e6 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:19.699239 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23163b4b-92dc-4c14-a022-5d90b95879e6 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:19.700096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23163b4b-92dc-4c14-a022-5d90b95879e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:19.724397 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-23163b4b-92dc-4c14-a022-5d90b95879e6 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:19.725605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-23163b4b-92dc-4c14-a022-5d90b95879e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:19.736709 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad25e1f-2ce5-460b-9ab7-6e7d89155126 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:19.770863 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ff7e3-8ba0-4ef3-86d7-38f3848037eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:19.771804 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ff7e3-8ba0-4ef3-86d7-38f3848037eb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:19.772226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ff7e3-8ba0-4ef3-86d7-38f3848037eb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:19.772635 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ff7e3-8ba0-4ef3-86d7-38f3848037eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:19.798068 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 435/847] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 04:47:19.823512 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23163b4b-92dc-4c14-a022-5d90b95879e6 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:19.861120 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ff7e3-8ba0-4ef3-86d7-38f3848037eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:19.862987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ff7e3-8ba0-4ef3-86d7-38f3848037eb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:19.888578 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:19.930672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23163b4b-92dc-4c14-a022-5d90b95879e6 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:19.945767 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:19.968012 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148358a0-8997-45c7-8244-98da7b7a20cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:19.968754 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148358a0-8997-45c7-8244-98da7b7a20cb None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:19.969039 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148358a0-8997-45c7-8244-98da7b7a20cb None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:19.969312 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148358a0-8997-45c7-8244-98da7b7a20cb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:19.970140 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148358a0-8997-45c7-8244-98da7b7a20cb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:19.971427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148358a0-8997-45c7-8244-98da7b7a20cb None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:19.974387 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 413/848] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:47:19 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 04:47:19.983757 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ff7e3-8ba0-4ef3-86d7-38f3848037eb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:19.985561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-148358a0-8997-45c7-8244-98da7b7a20cb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 04:47:19.985561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:19.985561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 04:47:19.985561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 04:47:19.985561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 04:47:19.985561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 04:47:19.985561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 04:47:19.985561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:19.985561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 04:47:19.985561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:19.985561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:47:19.985561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:19.985561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:19.985561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:19.985561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:19.985561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:19.986694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:19.986694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:19.986694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:19.986694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:19.986694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:19.986694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:19.986694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:19.986694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 04:47:19.986694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 04:47:19.986694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 04:47:19.986694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 04:47:19.986694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:19.986694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:19.986694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:19.986694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:19.986694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:19.987538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:19.987538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:19.987538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:19.987538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:19.987538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:19.987538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:19.987538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:19.987538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:19.987538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:19.987538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:19.987538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:19.987538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 04:47:19.987538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:19.987538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 04:47:19.987538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:19.991470 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 436/849] 10.222.0.17 () {56 vars in 1132 bytes} [Tue Jul 26 04:47:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:19.992644 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148358a0-8997-45c7-8244-98da7b7a20cb None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:20.103075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148358a0-8997-45c7-8244-98da7b7a20cb None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:20.111993 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ff7e3-8ba0-4ef3-86d7-38f3848037eb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:20.136464 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:20.142546 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92563bda-55c3-4899-a4de-f1c5fed11340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:20.143469 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92563bda-55c3-4899-a4de-f1c5fed11340 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:20.143838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92563bda-55c3-4899-a4de-f1c5fed11340 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:20.144174 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92563bda-55c3-4899-a4de-f1c5fed11340 None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:20.145056 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92563bda-55c3-4899-a4de-f1c5fed11340 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:20.145664 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92563bda-55c3-4899-a4de-f1c5fed11340 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:20.146986 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92563bda-55c3-4899-a4de-f1c5fed11340 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:20.150677 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e5fd1-12a5-4eb2-9a22-82abb107c666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:20.151524 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e5fd1-12a5-4eb2-9a22-82abb107c666 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:20.151994 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e5fd1-12a5-4eb2-9a22-82abb107c666 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:20.152418 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e5fd1-12a5-4eb2-9a22-82abb107c666 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:20.157060 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 437/850] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Jul 26 04:47:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:20.158034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92563bda-55c3-4899-a4de-f1c5fed11340 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:20.163943 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96e5fd1-12a5-4eb2-9a22-82abb107c666 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:20.165617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96e5fd1-12a5-4eb2-9a22-82abb107c666 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:20.181097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a96e5fd1-12a5-4eb2-9a22-82abb107c666 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:20.181097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:20.181097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:20.181097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:20.181097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:20.181097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:20.181097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:20.181097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:20.181097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:20.181097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:20.181097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:20.181097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:20.181097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:20.181097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:20.181097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:20.182328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:20.182328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:20.182328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:20.182328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:20.182328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:20.182328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:20.182328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:20.182328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:20.182328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:20.182328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:20.182328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:20.182328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:20.182328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:20.182328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:20.182328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:20.182328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:20.182328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:20.183530 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:20.183530 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:20.183530 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:20.183530 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:20.189971 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 414/851] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:47:20.191120 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e5fd1-12a5-4eb2-9a22-82abb107c666 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:20.212115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:20.267779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92563bda-55c3-4899-a4de-f1c5fed11340 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:20.295406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e5fd1-12a5-4eb2-9a22-82abb107c666 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:20.304157 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ee137-2097-4407-b3c7-f09d9cdfe4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:20.305068 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ee137-2097-4407-b3c7-f09d9cdfe4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:20.305471 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ee137-2097-4407-b3c7-f09d9cdfe4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:20.305861 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ee137-2097-4407-b3c7-f09d9cdfe4e2 None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:20.306643 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ee137-2097-4407-b3c7-f09d9cdfe4e2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:20.307096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ee137-2097-4407-b3c7-f09d9cdfe4e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:20.308903 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ee137-2097-4407-b3c7-f09d9cdfe4e2 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:20.320085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 438/852] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Jul 26 04:47:20 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:20.337331 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f307288-e30e-428a-809e-2fafb1b73001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:20.338431 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f307288-e30e-428a-809e-2fafb1b73001 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:20.338872 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f307288-e30e-428a-809e-2fafb1b73001 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:20.339319 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f307288-e30e-428a-809e-2fafb1b73001 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:20.340621 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f307288-e30e-428a-809e-2fafb1b73001 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:20.341077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f307288-e30e-428a-809e-2fafb1b73001 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:20.342668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f307288-e30e-428a-809e-2fafb1b73001 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:20.360551 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 415/853] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:20.366628 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ee137-2097-4407-b3c7-f09d9cdfe4e2 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:20.488058 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ee137-2097-4407-b3c7-f09d9cdfe4e2 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:20.496442 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:20.517674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286a096-b6f0-4d65-aaf2-bda87b3a7720 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:20.518017 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286a096-b6f0-4d65-aaf2-bda87b3a7720 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:20.518222 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286a096-b6f0-4d65-aaf2-bda87b3a7720 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:20.519882 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 416/854] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:47:20.526538 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e6a6c-31c1-47fa-83a6-0948cdd10b6d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:20.527048 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e6a6c-31c1-47fa-83a6-0948cdd10b6d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:20.527537 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e6a6c-31c1-47fa-83a6-0948cdd10b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:20.530225 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e6a46-2fe7-40e0-bf07-9d8f7733e5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:20.530834 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e6a46-2fe7-40e0-bf07-9d8f7733e5e0 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:20.531118 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e6a46-2fe7-40e0-bf07-9d8f7733e5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:20.531409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e6a46-2fe7-40e0-bf07-9d8f7733e5e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:20.540190 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3e6a46-2fe7-40e0-bf07-9d8f7733e5e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:20.542334 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3e6a46-2fe7-40e0-bf07-9d8f7733e5e0 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:20.547983 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f48e6a6c-31c1-47fa-83a6-0948cdd10b6d None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:20.549194 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f48e6a6c-31c1-47fa-83a6-0948cdd10b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:20.554530 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-dd3e6a46-2fe7-40e0-bf07-9d8f7733e5e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:20.554530 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:20.554530 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:20.554530 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:20.554530 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:20.554530 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:20.554530 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:20.554530 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:20.554530 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:20.554530 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:20.554530 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:20.554530 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:20.554530 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:47:20.554530 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:47:20.554530 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:47:20.555438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:47:20.555438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:20.555438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:20.555438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:20.555438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:20.555438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:20.555438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:20.555438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:20.555438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:20.555438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:20.555438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:20.555438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:20.555438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:20.555438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:20.555438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:20.555438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:20.555438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:20.556364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:47:20.556364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:47:20.556364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:47:20.556364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:47:20.556364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:47:20.556364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:47:20.556364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:20.556364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:20.556364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 439/855] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:47:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:20.561856 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e6a46-2fe7-40e0-bf07-9d8f7733e5e0 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:20.598557 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:20.663522 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 417/856] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 04:47:20.678978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e6a6c-31c1-47fa-83a6-0948cdd10b6d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:20.694567 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e6a46-2fe7-40e0-bf07-9d8f7733e5e0 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:20.732023 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee520c-fa9b-4052-8fbc-955d2b99ec5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:20.733184 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee520c-fa9b-4052-8fbc-955d2b99ec5a None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:20.733786 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee520c-fa9b-4052-8fbc-955d2b99ec5a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:20.734064 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee520c-fa9b-4052-8fbc-955d2b99ec5a None admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:20.734846 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee520c-fa9b-4052-8fbc-955d2b99ec5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:20.735159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee520c-fa9b-4052-8fbc-955d2b99ec5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:20.736669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee520c-fa9b-4052-8fbc-955d2b99ec5a None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:20.750753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 440/857] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:47:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:20.751858 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bca060-c406-4acc-a991-75d461211552 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:20.752149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bca060-c406-4acc-a991-75d461211552 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:20.752433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bca060-c406-4acc-a991-75d461211552 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:20.757986 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 441/858] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:20.757986 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bca060-c406-4acc-a991-75d461211552 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:20.784488 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:20.806161 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e6a6c-31c1-47fa-83a6-0948cdd10b6d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:20.851191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f5a2b-1ed1-411b-ab57-733f8da4a36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:20.852110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f5a2b-1ed1-411b-ab57-733f8da4a36a None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:20.852418 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f5a2b-1ed1-411b-ab57-733f8da4a36a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:20.853088 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f5a2b-1ed1-411b-ab57-733f8da4a36a None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:20.854142 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f5a2b-1ed1-411b-ab57-733f8da4a36a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:20.856086 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f5a2b-1ed1-411b-ab57-733f8da4a36a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:20.863303 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bca060-c406-4acc-a991-75d461211552 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:20.877959 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 418/859] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jul 26 04:47:20 2022] POST /identity/v3/services => generated 251 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 04:47:20.888922 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460e59e-c1b9-4f75-8353-4faa3265ec23 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:20.889165 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460e59e-c1b9-4f75-8353-4faa3265ec23 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:20.889406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460e59e-c1b9-4f75-8353-4faa3265ec23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:20.916774 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3322e-8176-4511-a1d3-baa20f876a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:20.917408 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3322e-8176-4511-a1d3-baa20f876a32 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:20.917694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3322e-8176-4511-a1d3-baa20f876a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:20.917962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3322e-8176-4511-a1d3-baa20f876a32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:20.918830 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c3322e-8176-4511-a1d3-baa20f876a32 None admin] RBAC: Adding query filter params (role.id=72bf8bd23d2c4b76ad32b7b35d6db412, scope.project.id=d583e0be8ef74766bffd6c3c07af18f1, user.id=d5218319aaa5424882714bffcbeee730) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:20.919180 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c3322e-8176-4511-a1d3-baa20f876a32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:20.920801 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c3322e-8176-4511-a1d3-baa20f876a32 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:20.925084 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d460e59e-c1b9-4f75-8353-4faa3265ec23 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:20.927054 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d460e59e-c1b9-4f75-8353-4faa3265ec23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:20.936105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 442/860] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:47:20 2022] GET /identity/v3/role_assignments?user.id=d5218319aaa5424882714bffcbeee730&scope.project.id=d583e0be8ef74766bffd6c3c07af18f1&role.id=72bf8bd23d2c4b76ad32b7b35d6db412 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:20.950842 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:21.020613 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 419/861] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:20 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:21.043715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3322e-8176-4511-a1d3-baa20f876a32 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:21.146546 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:21.164039 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3322e-8176-4511-a1d3-baa20f876a32 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:21.200315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf449964-4a57-4c77-ad26-5cad92623eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:21.200945 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf449964-4a57-4c77-ad26-5cad92623eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:21.201224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf449964-4a57-4c77-ad26-5cad92623eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:21.201607 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf449964-4a57-4c77-ad26-5cad92623eb5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:21.213071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf449964-4a57-4c77-ad26-5cad92623eb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:47:21.218774 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-cf449964-4a57-4c77-ad26-5cad92623eb5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:47:21.218774 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:21.218774 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:21.218774 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:21.218774 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:21.218774 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:21.218774 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:21.218774 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:21.218774 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:21.218774 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:21.218774 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:21.218774 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:21.218774 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:47:21.218774 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:47:21.218774 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:47:21.218774 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:47:21.219789 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:47:21.219789 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:47:21.219789 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:47:21.219789 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:47:21.219789 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:21.219789 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:21.219789 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:21.219789 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:21.219789 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:21.219789 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:21.219789 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:21.219789 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:21.219789 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:21.219789 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:21.219789 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:21.219789 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:21.220757 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:21.220757 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:21.220757 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:21.220757 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:21.220757 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:47:21.220757 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:47:21.220757 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:47:21.220757 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:47:21.220757 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:47:21.220757 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:47:21.220757 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:47:21.220757 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:21.220757 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 443/862] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jul 26 04:47:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:21.227989 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d460e59e-c1b9-4f75-8353-4faa3265ec23 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:21.335124 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d460e59e-c1b9-4f75-8353-4faa3265ec23 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:21.375629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaed6b8-e9e2-4624-b295-3e893070fc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:21.376583 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaed6b8-e9e2-4624-b295-3e893070fc40 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:21.377062 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaed6b8-e9e2-4624-b295-3e893070fc40 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:21.377502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaed6b8-e9e2-4624-b295-3e893070fc40 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:21.378463 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaed6b8-e9e2-4624-b295-3e893070fc40 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:21.378941 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaed6b8-e9e2-4624-b295-3e893070fc40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:21.380538 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaed6b8-e9e2-4624-b295-3e893070fc40 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:21.394795 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 420/863] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jul 26 04:47:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:21.396898 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf449964-4a57-4c77-ad26-5cad92623eb5 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:21.509263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf449964-4a57-4c77-ad26-5cad92623eb5 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:21.547951 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66852307-297c-43f1-b46e-04f5c043e7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:21.548609 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66852307-297c-43f1-b46e-04f5c043e7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:21.548892 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66852307-297c-43f1-b46e-04f5c043e7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:21.549163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66852307-297c-43f1-b46e-04f5c043e7c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:21.560861 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66852307-297c-43f1-b46e-04f5c043e7c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:21.562340 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66852307-297c-43f1-b46e-04f5c043e7c1 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:21.569570 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:21.578255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-66852307-297c-43f1-b46e-04f5c043e7c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:21.578255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:21.578255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:21.578255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:21.578255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:21.578255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:21.578255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:21.578255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:21.578255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:21.578255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:21.578255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:21.578255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:21.578255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:21.578255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:21.578255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:21.579148 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:21.579148 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:21.579148 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:21.579148 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:21.579148 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:21.579148 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:21.579148 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:21.579148 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:21.579148 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:21.579148 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:21.579148 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:21.579148 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:21.579148 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:21.579148 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:21.579148 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:21.579148 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:21.579148 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:21.580073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:21.580073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:21.580073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:21.580073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:21.583904 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 444/864] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:21.585752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaed6b8-e9e2-4624-b295-3e893070fc40 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:21.606369 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:21.696023 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaed6b8-e9e2-4624-b295-3e893070fc40 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:21.732262 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5116e4-7d3a-4f0d-8589-1f8b797d015c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:21.732900 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5116e4-7d3a-4f0d-8589-1f8b797d015c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:21.733182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5116e4-7d3a-4f0d-8589-1f8b797d015c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:21.733461 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5116e4-7d3a-4f0d-8589-1f8b797d015c None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:21.734155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5116e4-7d3a-4f0d-8589-1f8b797d015c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:21.734457 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5116e4-7d3a-4f0d-8589-1f8b797d015c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:21.735847 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5116e4-7d3a-4f0d-8589-1f8b797d015c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:21.753535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 421/865] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:21.754913 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66852307-297c-43f1-b46e-04f5c043e7c1 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:21.865148 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66852307-297c-43f1-b46e-04f5c043e7c1 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:21.898905 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e39380-28b2-42e4-a52c-43c832408943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:21.899568 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e39380-28b2-42e4-a52c-43c832408943 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:21.899895 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e39380-28b2-42e4-a52c-43c832408943 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:21.900171 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e39380-28b2-42e4-a52c-43c832408943 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:21.907952 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e39380-28b2-42e4-a52c-43c832408943 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:21.909410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e39380-28b2-42e4-a52c-43c832408943 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:21.920720 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-47e39380-28b2-42e4-a52c-43c832408943 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 04:47:21.920720 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:21.920720 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:21.920720 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:21.920720 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:21.920720 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:21.920720 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:21.920720 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:21.920720 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:21.920720 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:21.920720 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:21.920720 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:21.920720 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:47:21.920720 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:47:21.920720 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:47:21.920720 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:47:21.921669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:21.921669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:21.921669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:47:21.921669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:21.921669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:47:21.921669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:21.921669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:21.921669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:21.921669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:21.921669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:21.921669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:21.921669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:21.921669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:21.921669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:21.921669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:21.921669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:21.921669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:21.921669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:21.922605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:21.922605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:21.922605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:47:21.922605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:47:21.922605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:47:21.922605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:47:21.922605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:47:21.922605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:47:21.922605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 04:47:21.922605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:21.926631 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 445/866] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Jul 26 04:47:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:21.928041 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5116e4-7d3a-4f0d-8589-1f8b797d015c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:22.011717 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:22.035725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5116e4-7d3a-4f0d-8589-1f8b797d015c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:22.073696 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaaa76e-4b25-41e6-a6fd-5206ef8b2ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:22.074337 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaaa76e-4b25-41e6-a6fd-5206ef8b2ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:22.074619 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaaa76e-4b25-41e6-a6fd-5206ef8b2ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:22.074891 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaaa76e-4b25-41e6-a6fd-5206ef8b2ef1 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:22.075729 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaaa76e-4b25-41e6-a6fd-5206ef8b2ef1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:22.076036 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaaa76e-4b25-41e6-a6fd-5206ef8b2ef1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:22.077540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaaa76e-4b25-41e6-a6fd-5206ef8b2ef1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:22.102974 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 422/867] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Jul 26 04:47:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:22.104530 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e39380-28b2-42e4-a52c-43c832408943 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:22.167939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:22.214119 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e39380-28b2-42e4-a52c-43c832408943 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:22.250600 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c0fd80-4f6e-4611-8937-279d0b79192f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:22.251350 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0fd80-4f6e-4611-8937-279d0b79192f None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:22.251662 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0fd80-4f6e-4611-8937-279d0b79192f None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:22.251939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0fd80-4f6e-4611-8937-279d0b79192f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:22.322223 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c0fd80-4f6e-4611-8937-279d0b79192f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:22.323796 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c0fd80-4f6e-4611-8937-279d0b79192f None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:22.387011 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:22.431841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 446/868] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:47:22 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 04:47:22.436203 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaaa76e-4b25-41e6-a6fd-5206ef8b2ef1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:22.466221 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427c18e-8326-474c-b958-d8cced6a4c1b None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:22.466538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427c18e-8326-474c-b958-d8cced6a4c1b None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:22.466758 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427c18e-8326-474c-b958-d8cced6a4c1b None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:22.468511 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 447/869] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:47:22.473832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdd2d4-dcb8-48bc-9b90-39e3bb68be61 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:22.474121 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdd2d4-dcb8-48bc-9b90-39e3bb68be61 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:22.474497 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdd2d4-dcb8-48bc-9b90-39e3bb68be61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:22.493815 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-66fdd2d4-dcb8-48bc-9b90-39e3bb68be61 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:22.495174 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-66fdd2d4-dcb8-48bc-9b90-39e3bb68be61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:22.575259 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaaa76e-4b25-41e6-a6fd-5206ef8b2ef1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:22.583671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 448/870] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:22 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 04:47:22.603626 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdd2d4-dcb8-48bc-9b90-39e3bb68be61 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:22.619147 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82efb9d-fb05-409d-aed3-310cce5fe117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:22.620039 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82efb9d-fb05-409d-aed3-310cce5fe117 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:22.620329 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82efb9d-fb05-409d-aed3-310cce5fe117 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:22.620993 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82efb9d-fb05-409d-aed3-310cce5fe117 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:22.623539 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:22.635699 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82efb9d-fb05-409d-aed3-310cce5fe117 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:22.637787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82efb9d-fb05-409d-aed3-310cce5fe117 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:22.657224 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b82efb9d-fb05-409d-aed3-310cce5fe117 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:22.657224 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:22.657224 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:22.657224 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:22.657224 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:22.657224 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:22.657224 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:22.657224 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:22.657224 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:22.657224 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:22.657224 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:22.657224 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:22.657224 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:22.657224 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:22.657224 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:22.658168 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:22.658168 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:22.658168 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:22.658168 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:22.658168 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:22.658168 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:22.658168 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:22.658168 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:22.658168 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:22.658168 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:22.658168 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:22.658168 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:22.658168 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:22.658168 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:22.658168 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:22.658168 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:22.658168 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:22.659051 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:22.659051 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:22.659051 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:22.659051 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:22.663658 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 423/871] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:22.666429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82efb9d-fb05-409d-aed3-310cce5fe117 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:22.713708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:22.729182 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdd2d4-dcb8-48bc-9b90-39e3bb68be61 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:22.769930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc24b1-4066-460a-9616-62b31a0340c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:22.770571 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc24b1-4066-460a-9616-62b31a0340c3 None admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:22.770912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc24b1-4066-460a-9616-62b31a0340c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:22.771190 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc24b1-4066-460a-9616-62b31a0340c3 None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:22.772331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc24b1-4066-460a-9616-62b31a0340c3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:22.774781 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc24b1-4066-460a-9616-62b31a0340c3 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:22.798869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82efb9d-fb05-409d-aed3-310cce5fe117 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:22.800109 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 449/872] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jul 26 04:47:22 2022] POST /identity/v3/services => generated 269 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 04:47:22.867694 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66425ce-f207-42ac-a4f3-8c24331bf119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:22.868756 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66425ce-f207-42ac-a4f3-8c24331bf119 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:22.869322 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66425ce-f207-42ac-a4f3-8c24331bf119 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:22.869799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66425ce-f207-42ac-a4f3-8c24331bf119 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:22.870743 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66425ce-f207-42ac-a4f3-8c24331bf119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:22.871262 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66425ce-f207-42ac-a4f3-8c24331bf119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:22.872934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66425ce-f207-42ac-a4f3-8c24331bf119 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:22.891692 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 424/873] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:22.893922 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc24b1-4066-460a-9616-62b31a0340c3 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:23.034910 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc24b1-4066-460a-9616-62b31a0340c3 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:23.035515 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee23cfc-c86b-45f2-97aa-4d9e67a5911e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:23.035871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee23cfc-c86b-45f2-97aa-4d9e67a5911e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:23.036149 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee23cfc-c86b-45f2-97aa-4d9e67a5911e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:23.037786 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 425/874] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:47:23.043211 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53976636-d827-4226-9965-0fad78078202 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:23.043495 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53976636-d827-4226-9965-0fad78078202 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:23.043807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53976636-d827-4226-9965-0fad78078202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:23.070156 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-53976636-d827-4226-9965-0fad78078202 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:23.070591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-53976636-d827-4226-9965-0fad78078202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:23.079089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e74c5a-d920-4c96-9526-a9116f7e99f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:23.079302 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e74c5a-d920-4c96-9526-a9116f7e99f9 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:23.079991 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e74c5a-d920-4c96-9526-a9116f7e99f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:23.079991 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e74c5a-d920-4c96-9526-a9116f7e99f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:23.087544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e74c5a-d920-4c96-9526-a9116f7e99f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:23.089138 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e74c5a-d920-4c96-9526-a9116f7e99f9 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:23.102001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-65e74c5a-d920-4c96-9526-a9116f7e99f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:23.102001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:23.102001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:23.102001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:23.102001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:23.102001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:23.102001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:23.102001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:23.102001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:23.102001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:23.102001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:23.102001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:23.102001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:47:23.102001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:47:23.102001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:47:23.103034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:47:23.103034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:23.103034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:23.103034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:23.103034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:23.103034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:23.103034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:23.103034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:23.103034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:23.103034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:23.103034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:23.103034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:23.103034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:23.103034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:23.103034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:23.103034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:23.103034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:23.103968 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:47:23.103968 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:47:23.103968 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:47:23.103968 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:47:23.103968 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:47:23.103968 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:47:23.103968 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:23.103968 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:23.103968 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 450/875] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:47:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:23.109344 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e74c5a-d920-4c96-9526-a9116f7e99f9 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:23.115160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:23.143964 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:23.152358 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 426/876] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:23.153970 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d8795-97fc-47ab-8f62-fb5acfe0a7e0 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:23.154264 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d8795-97fc-47ab-8f62-fb5acfe0a7e0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:23.154544 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d8795-97fc-47ab-8f62-fb5acfe0a7e0 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:23.160136 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 427/877] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:23.169952 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e242a99-07e8-431c-8915-bad9cc1e292d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:23.170262 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e242a99-07e8-431c-8915-bad9cc1e292d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:23.170511 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e242a99-07e8-431c-8915-bad9cc1e292d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:23.190832 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2e242a99-07e8-431c-8915-bad9cc1e292d None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:23.192092 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2e242a99-07e8-431c-8915-bad9cc1e292d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:23.240979 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e74c5a-d920-4c96-9526-a9116f7e99f9 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:23.271834 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 428/878] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:23.271834 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e242a99-07e8-431c-8915-bad9cc1e292d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:23.296281 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2e274-4a90-4f5a-8c92-cfc5833a2c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:23.297916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2e274-4a90-4f5a-8c92-cfc5833a2c43 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:23.298399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2e274-4a90-4f5a-8c92-cfc5833a2c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:23.298821 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2e274-4a90-4f5a-8c92-cfc5833a2c43 None admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:23.299794 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c2e274-4a90-4f5a-8c92-cfc5833a2c43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:23.300290 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c2e274-4a90-4f5a-8c92-cfc5833a2c43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:23.302824 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c2e274-4a90-4f5a-8c92-cfc5833a2c43 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:23.321090 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 451/879] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:47:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:23.322693 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2e274-4a90-4f5a-8c92-cfc5833a2c43 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:23.390657 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e242a99-07e8-431c-8915-bad9cc1e292d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:23.392813 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:23.402502 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:23.433136 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ddbf8-6586-4533-a0ae-ccafa5ebca78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:23.433810 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ddbf8-6586-4533-a0ae-ccafa5ebca78 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:23.434103 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ddbf8-6586-4533-a0ae-ccafa5ebca78 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:23.434595 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ddbf8-6586-4533-a0ae-ccafa5ebca78 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:23.435023 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ddbf8-6586-4533-a0ae-ccafa5ebca78 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:23.436476 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ddbf8-6586-4533-a0ae-ccafa5ebca78 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:23.437493 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2e274-4a90-4f5a-8c92-cfc5833a2c43 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:23.446562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-bf5ddbf8-6586-4533-a0ae-ccafa5ebca78 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 04:47:23.446562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:23.446562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 04:47:23.446562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 04:47:23.446562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 04:47:23.446562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 04:47:23.446562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 04:47:23.446562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:23.446562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 04:47:23.446562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:23.446562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:47:23.446562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:23.446562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:23.446562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:23.446562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:23.446562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:23.446562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:23.448021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:23.448021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:23.448021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:23.448021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:23.448021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:23.448021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:23.448021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 04:47:23.448021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 04:47:23.448021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 04:47:23.448021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 04:47:23.448021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:23.448021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:23.448021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:23.448021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:23.448021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:23.448021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:23.448860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:23.448860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:23.448860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:23.448860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:23.448860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:23.448860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:23.448860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:23.448860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:23.448860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:23.448860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:23.448860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 04:47:23.448860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:23.448860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 04:47:23.448860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:23.448860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 429/880] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Jul 26 04:47:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:23.449645 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ddbf8-6586-4533-a0ae-ccafa5ebca78 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:23.480704 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85eaa6-f439-4a10-b287-86831509c9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:23.481444 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85eaa6-f439-4a10-b287-86831509c9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:23.481648 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85eaa6-f439-4a10-b287-86831509c9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:23.481909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85eaa6-f439-4a10-b287-86831509c9c9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:23.482744 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85eaa6-f439-4a10-b287-86831509c9c9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:23.483980 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85eaa6-f439-4a10-b287-86831509c9c9 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:23.494018 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-1f85eaa6-f439-4a10-b287-86831509c9c9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 04:47:23.494018 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:23.494018 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 04:47:23.494018 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 04:47:23.494018 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 04:47:23.494018 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 04:47:23.494018 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 04:47:23.494018 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:23.494018 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 04:47:23.494018 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:23.494018 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:47:23.494018 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:23.494018 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:23.494018 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:23.494018 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:23.494018 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:23.494018 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:23.495216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:23.495216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:23.495216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:23.495216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:23.495216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:23.495216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:23.495216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 04:47:23.495216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 04:47:23.495216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 04:47:23.495216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 04:47:23.495216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:23.495216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:23.495216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:23.495216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:23.495216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:23.495216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:23.496071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:23.496071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:23.496071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:23.496071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:23.496071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:23.496071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:23.496071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:23.496071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:23.496071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:23.496071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:23.496071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 04:47:23.496071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:23.496071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 04:47:23.496071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:23.496071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 452/881] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jul 26 04:47:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:23.496908 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85eaa6-f439-4a10-b287-86831509c9c9 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:23.561687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ddbf8-6586-4533-a0ae-ccafa5ebca78 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:23.598282 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2358e1-fe97-426f-a5c7-86fa2dd2c52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:23.598922 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2358e1-fe97-426f-a5c7-86fa2dd2c52d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:23.599202 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2358e1-fe97-426f-a5c7-86fa2dd2c52d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:23.599627 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2358e1-fe97-426f-a5c7-86fa2dd2c52d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:23.600515 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2358e1-fe97-426f-a5c7-86fa2dd2c52d None admin] RBAC: Adding query filter params (role.id=e5048f4e7c6b4494a15d628758090380, scope.project.id=d583e0be8ef74766bffd6c3c07af18f1, user.id=3a8f3eb811ae40ad9cb64261f95a992e) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:23.600833 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2358e1-fe97-426f-a5c7-86fa2dd2c52d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:23.602302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2358e1-fe97-426f-a5c7-86fa2dd2c52d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:23.609096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 430/882] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:47:23 2022] GET /identity/v3/role_assignments?user.id=3a8f3eb811ae40ad9cb64261f95a992e&scope.project.id=d583e0be8ef74766bffd6c3c07af18f1&role.id=e5048f4e7c6b4494a15d628758090380 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:23.609778 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85eaa6-f439-4a10-b287-86831509c9c9 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:23.609925 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2358e1-fe97-426f-a5c7-86fa2dd2c52d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:23.663513 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:23.664811 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:23.671782 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f332b-8ed4-44b3-a15a-b030fbb0e3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:23.672487 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f332b-8ed4-44b3-a15a-b030fbb0e3ab None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:23.672795 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f332b-8ed4-44b3-a15a-b030fbb0e3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:23.674000 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f332b-8ed4-44b3-a15a-b030fbb0e3ab None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:23.674883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f332b-8ed4-44b3-a15a-b030fbb0e3ab None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:23.675192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f332b-8ed4-44b3-a15a-b030fbb0e3ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:23.676678 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f332b-8ed4-44b3-a15a-b030fbb0e3ab None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:23.688016 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 453/883] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Jul 26 04:47:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:23.689071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f332b-8ed4-44b3-a15a-b030fbb0e3ab None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:23.752887 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2358e1-fe97-426f-a5c7-86fa2dd2c52d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:23.793405 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81153a08-e047-4de2-aa71-41d454701dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:23.794391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81153a08-e047-4de2-aa71-41d454701dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:23.794866 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81153a08-e047-4de2-aa71-41d454701dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:23.795361 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81153a08-e047-4de2-aa71-41d454701dd4 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:23.796369 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81153a08-e047-4de2-aa71-41d454701dd4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:23.797038 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81153a08-e047-4de2-aa71-41d454701dd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:23.798726 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81153a08-e047-4de2-aa71-41d454701dd4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:23.805183 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 431/884] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jul 26 04:47:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:23.810635 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81153a08-e047-4de2-aa71-41d454701dd4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:23.821926 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f332b-8ed4-44b3-a15a-b030fbb0e3ab None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:23.862738 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1756cf-f614-41ec-8aeb-2754b84aea31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:23.863781 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1756cf-f614-41ec-8aeb-2754b84aea31 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:23.864263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1756cf-f614-41ec-8aeb-2754b84aea31 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:23.864744 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1756cf-f614-41ec-8aeb-2754b84aea31 None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:23.865441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1756cf-f614-41ec-8aeb-2754b84aea31 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:23.865745 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1756cf-f614-41ec-8aeb-2754b84aea31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:23.867137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1756cf-f614-41ec-8aeb-2754b84aea31 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:23.874350 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 454/885] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Jul 26 04:47:23 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:23.936751 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:23.945963 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81153a08-e047-4de2-aa71-41d454701dd4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:23.985774 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f61304-0b3b-46c6-acc6-189b54a0cf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:23.986430 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f61304-0b3b-46c6-acc6-189b54a0cf98 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:23.986712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f61304-0b3b-46c6-acc6-189b54a0cf98 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:23.986983 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f61304-0b3b-46c6-acc6-189b54a0cf98 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:23.987721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f61304-0b3b-46c6-acc6-189b54a0cf98 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:23.988033 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f61304-0b3b-46c6-acc6-189b54a0cf98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:23.989478 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f61304-0b3b-46c6-acc6-189b54a0cf98 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:23.996095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 432/886] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jul 26 04:47:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:24.002218 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1756cf-f614-41ec-8aeb-2754b84aea31 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:24.078288 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:24.124182 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1756cf-f614-41ec-8aeb-2754b84aea31 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:24.158482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf46ee-202f-46f0-bf2f-37b3bfc98758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:24.159110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf46ee-202f-46f0-bf2f-37b3bfc98758 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:24.159470 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf46ee-202f-46f0-bf2f-37b3bfc98758 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:24.159958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf46ee-202f-46f0-bf2f-37b3bfc98758 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:24.161287 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cf46ee-202f-46f0-bf2f-37b3bfc98758 None admin] RBAC: Adding query filter params (interface=public, service_id=96f6b27f4ae94e349a1b7d4da36dbea1, region_id=RegionOne) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:24.161848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cf46ee-202f-46f0-bf2f-37b3bfc98758 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:24.164515 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cf46ee-202f-46f0-bf2f-37b3bfc98758 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:24.171672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 455/887] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Jul 26 04:47:24 2022] GET /identity/v3/endpoints?service_id=96f6b27f4ae94e349a1b7d4da36dbea1&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:24.177282 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f61304-0b3b-46c6-acc6-189b54a0cf98 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:24.284026 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f61304-0b3b-46c6-acc6-189b54a0cf98 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:24.320432 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed984203-03a0-4ac1-a4c7-9e77b45dcf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:24.321074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed984203-03a0-4ac1-a4c7-9e77b45dcf0d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:24.321401 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed984203-03a0-4ac1-a4c7-9e77b45dcf0d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:24.321759 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed984203-03a0-4ac1-a4c7-9e77b45dcf0d None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:24.322486 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed984203-03a0-4ac1-a4c7-9e77b45dcf0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:24.323888 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed984203-03a0-4ac1-a4c7-9e77b45dcf0d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:24.330451 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 433/888] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Jul 26 04:47:24 2022] GET /identity/v3/services => generated 1268 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 04:47:24.948907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b00f77-9a74-4bc6-9216-935c31459280 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:24.949550 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b00f77-9a74-4bc6-9216-935c31459280 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:24.949550 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b00f77-9a74-4bc6-9216-935c31459280 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:24.951056 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 456/889] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:24.956742 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46045984-1c21-4dcf-9ff1-de177cfd1c02 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:24.957217 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46045984-1c21-4dcf-9ff1-de177cfd1c02 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:24.957299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46045984-1c21-4dcf-9ff1-de177cfd1c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:24.976734 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-46045984-1c21-4dcf-9ff1-de177cfd1c02 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:24.977944 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-46045984-1c21-4dcf-9ff1-de177cfd1c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:24.996095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:25.051032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 434/890] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:25.074009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b00f77-9a74-4bc6-9216-935c31459280 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:25.185417 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b00f77-9a74-4bc6-9216-935c31459280 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:25.213743 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:25.225487 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c7b6f-0425-41cb-a373-60afd23d2912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:25.226141 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c7b6f-0425-41cb-a373-60afd23d2912 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:25.226427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c7b6f-0425-41cb-a373-60afd23d2912 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:25.226711 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c7b6f-0425-41cb-a373-60afd23d2912 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:25.227350 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c7b6f-0425-41cb-a373-60afd23d2912 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:25.228832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c7b6f-0425-41cb-a373-60afd23d2912 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:25.238059 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-7b4c7b6f-0425-41cb-a373-60afd23d2912 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 04:47:25.238059 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:25.238059 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 04:47:25.238059 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 04:47:25.238059 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 04:47:25.238059 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 04:47:25.238059 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 04:47:25.238059 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:25.238059 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 04:47:25.238059 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:25.238059 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:47:25.238059 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:25.238059 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:25.238059 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:25.238059 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:25.238059 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:25.239108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:25.239108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:25.239108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:25.239108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:25.239108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:25.239108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:25.239108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:25.239108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 04:47:25.239108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 04:47:25.239108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 04:47:25.239108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 04:47:25.239108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:25.239108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:25.239108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:25.239108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:25.239108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:25.240052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:25.240052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:25.240052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:25.240052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:25.240052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:25.240052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:25.240052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:25.240052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:25.240052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:25.240052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:25.240052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:25.240052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 04:47:25.240052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:25.240052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 04:47:25.240052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:25.240052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 457/891] 10.222.0.17 () {56 vars in 1132 bytes} [Tue Jul 26 04:47:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:47:25.247735 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46045984-1c21-4dcf-9ff1-de177cfd1c02 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:25.374370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46045984-1c21-4dcf-9ff1-de177cfd1c02 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:25.415208 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e0966-f678-4e38-a424-a610e392dee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:25.416354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e0966-f678-4e38-a424-a610e392dee8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:25.416846 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e0966-f678-4e38-a424-a610e392dee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:25.417323 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e0966-f678-4e38-a424-a610e392dee8 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:25.418255 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e0966-f678-4e38-a424-a610e392dee8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:25.418846 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e0966-f678-4e38-a424-a610e392dee8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:25.420523 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e0966-f678-4e38-a424-a610e392dee8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:25.426997 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 435/892] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Jul 26 04:47:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:25.435587 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c7b6f-0425-41cb-a373-60afd23d2912 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:25.550683 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c7b6f-0425-41cb-a373-60afd23d2912 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:25.587772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e636556e-83fa-4deb-a90f-7cdd871ff746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:25.588474 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636556e-83fa-4deb-a90f-7cdd871ff746 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:25.588759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636556e-83fa-4deb-a90f-7cdd871ff746 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:25.589035 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636556e-83fa-4deb-a90f-7cdd871ff746 None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:25.589784 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e636556e-83fa-4deb-a90f-7cdd871ff746 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:25.590456 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e636556e-83fa-4deb-a90f-7cdd871ff746 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:25.591907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e636556e-83fa-4deb-a90f-7cdd871ff746 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:25.597969 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 458/893] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Jul 26 04:47:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:25.604013 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e0966-f678-4e38-a424-a610e392dee8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:25.624907 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:25.712927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e0966-f678-4e38-a424-a610e392dee8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:25.753542 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46341b8-576e-44dc-87df-6c64a56bd276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:25.754186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46341b8-576e-44dc-87df-6c64a56bd276 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:25.754485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46341b8-576e-44dc-87df-6c64a56bd276 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:25.754824 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46341b8-576e-44dc-87df-6c64a56bd276 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:25.755717 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46341b8-576e-44dc-87df-6c64a56bd276 None admin] RBAC: Adding query filter params (interface=public, service_id=77f78e361b8a48dab709e041c23dc3b1, region_id=RegionOne) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:25.756104 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46341b8-576e-44dc-87df-6c64a56bd276 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:25.757556 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46341b8-576e-44dc-87df-6c64a56bd276 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:25.763889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 436/894] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Jul 26 04:47:25 2022] GET /identity/v3/endpoints?service_id=77f78e361b8a48dab709e041c23dc3b1&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:25.769930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e636556e-83fa-4deb-a90f-7cdd871ff746 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:25.831216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:25.848302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eba851-dcfe-41c6-98d8-6a0fb749a1a4 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:25.848610 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eba851-dcfe-41c6-98d8-6a0fb749a1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:25.848846 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eba851-dcfe-41c6-98d8-6a0fb749a1a4 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:25.850485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 437/895] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:47:25.857135 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522eb25-deec-43e7-b166-817ec6917a16 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:25.857437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522eb25-deec-43e7-b166-817ec6917a16 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:25.857697 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522eb25-deec-43e7-b166-817ec6917a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:25.879787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8522eb25-deec-43e7-b166-817ec6917a16 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:25.881623 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8522eb25-deec-43e7-b166-817ec6917a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:25.886154 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e636556e-83fa-4deb-a90f-7cdd871ff746 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:25.925101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc642b-8c5f-492d-874a-2483bc56ff7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:25.926101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc642b-8c5f-492d-874a-2483bc56ff7d None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:25.926461 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc642b-8c5f-492d-874a-2483bc56ff7d None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:25.926806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc642b-8c5f-492d-874a-2483bc56ff7d None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:25.927541 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc642b-8c5f-492d-874a-2483bc56ff7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:25.929035 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc642b-8c5f-492d-874a-2483bc56ff7d None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:25.939818 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 459/896] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Jul 26 04:47:25 2022] GET /identity/v3/services => generated 1268 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 04:47:25.968248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 438/897] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 04:47:25.991866 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc642b-8c5f-492d-874a-2483bc56ff7d None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:26.121006 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc642b-8c5f-492d-874a-2483bc56ff7d None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:26.139731 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:26.160977 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257b92e-964b-4281-b736-448eeb7f1a50 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:26.161323 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257b92e-964b-4281-b736-448eeb7f1a50 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:26.161543 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257b92e-964b-4281-b736-448eeb7f1a50 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:26.163208 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 439/898] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:26.166611 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c2b8c-5a8b-463e-a48a-d8997f92eb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:26.167668 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c2b8c-5a8b-463e-a48a-d8997f92eb20 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:26.168238 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c2b8c-5a8b-463e-a48a-d8997f92eb20 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:26.168731 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c2b8c-5a8b-463e-a48a-d8997f92eb20 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:26.168842 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee637f8-0982-438a-b9c3-8073568c5d61 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:26.169054 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee637f8-0982-438a-b9c3-8073568c5d61 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:26.169337 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee637f8-0982-438a-b9c3-8073568c5d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:26.181935 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c2b8c-5a8b-463e-a48a-d8997f92eb20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:47:26.188009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-c13c2b8c-5a8b-463e-a48a-d8997f92eb20 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 04:47:26.188009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:26.188009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:26.188009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:26.188009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:26.188009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:26.188009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:26.188009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:26.188009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:26.188009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:26.188009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:26.188009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:26.188009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:47:26.188009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:47:26.188009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:47:26.188009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:47:26.188983 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:47:26.188983 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:47:26.188983 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:47:26.188983 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:47:26.188983 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:26.188983 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:26.188983 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:26.188983 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:26.188983 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:26.188983 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:26.188983 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:26.188983 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:26.188983 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:26.188983 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:26.188983 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:26.188983 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:26.189877 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:26.189877 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:26.189877 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:26.189877 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:26.189877 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:47:26.189877 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:47:26.189877 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:47:26.189877 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:47:26.189877 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:47:26.189877 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:47:26.189877 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 04:47:26.189877 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:26.190519 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9ee637f8-0982-438a-b9c3-8073568c5d61 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:26.190519 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9ee637f8-0982-438a-b9c3-8073568c5d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:26.193184 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:26.193819 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 460/899] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Jul 26 04:47:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:26.198927 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c2b8c-5a8b-463e-a48a-d8997f92eb20 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:26.286461 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 440/900] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:26 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 04:47:26.308448 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee637f8-0982-438a-b9c3-8073568c5d61 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:26.319688 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c2b8c-5a8b-463e-a48a-d8997f92eb20 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:26.366022 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da74d14-015a-476f-833f-203b3a2b4c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:26.366654 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da74d14-015a-476f-833f-203b3a2b4c15 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:26.366937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da74d14-015a-476f-833f-203b3a2b4c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:26.367209 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da74d14-015a-476f-833f-203b3a2b4c15 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:26.367972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da74d14-015a-476f-833f-203b3a2b4c15 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:26.368393 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da74d14-015a-476f-833f-203b3a2b4c15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:26.369985 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da74d14-015a-476f-833f-203b3a2b4c15 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:26.380385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 461/901] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Jul 26 04:47:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:26.386259 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da74d14-015a-476f-833f-203b3a2b4c15 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:26.439049 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:26.445743 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee637f8-0982-438a-b9c3-8073568c5d61 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:26.495580 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c70c8b-798b-4759-a742-91a2b60e2568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:26.495580 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c70c8b-798b-4759-a742-91a2b60e2568 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:26.495580 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c70c8b-798b-4759-a742-91a2b60e2568 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:26.495580 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c70c8b-798b-4759-a742-91a2b60e2568 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:26.496024 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c70c8b-798b-4759-a742-91a2b60e2568 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:26.497567 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c70c8b-798b-4759-a742-91a2b60e2568 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:26.509441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da74d14-015a-476f-833f-203b3a2b4c15 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:26.510447 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:26.527444 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 441/902] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jul 26 04:47:26 2022] POST /identity/v3/services => generated 241 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:47:26.527444 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbb2a0-ec5a-4bfa-bc42-d6d0aa30ed23 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:26.527444 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbb2a0-ec5a-4bfa-bc42-d6d0aa30ed23 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:26.527444 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbb2a0-ec5a-4bfa-bc42-d6d0aa30ed23 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:26.527444 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 442/903] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:26.534053 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05635db0-e06a-422c-adb4-1f29578f7291 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:26.534400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05635db0-e06a-422c-adb4-1f29578f7291 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:26.534599 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05635db0-e06a-422c-adb4-1f29578f7291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:26.555474 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-05635db0-e06a-422c-adb4-1f29578f7291 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:26.556854 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-05635db0-e06a-422c-adb4-1f29578f7291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:26.566473 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da6a9e-bdb9-4795-8352-1d857b7671d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:26.567574 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da6a9e-bdb9-4795-8352-1d857b7671d1 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:26.568443 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da6a9e-bdb9-4795-8352-1d857b7671d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:26.569053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da6a9e-bdb9-4795-8352-1d857b7671d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:26.585951 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da6a9e-bdb9-4795-8352-1d857b7671d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:26.588566 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da6a9e-bdb9-4795-8352-1d857b7671d1 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:26.609460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-63da6a9e-bdb9-4795-8352-1d857b7671d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:26.609460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:26.609460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:26.609460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:26.609460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:26.609460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:26.609460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:26.609460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:26.609460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:26.609460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:26.609460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:26.609460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:26.609460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:26.609460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:26.609460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:26.610441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:26.610441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:26.610441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:26.610441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:26.610441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:26.610441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:26.610441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:26.610441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:26.610441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:26.610441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:26.610441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:26.610441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:26.610441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:26.610441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:26.610441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:26.610441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:26.610441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:26.611356 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:26.611356 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:26.611356 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:26.611356 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:26.613203 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 462/904] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:26.625341 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da6a9e-bdb9-4795-8352-1d857b7671d1 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:26.644910 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 443/905] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:26.667833 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05635db0-e06a-422c-adb4-1f29578f7291 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:26.746247 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:26.766550 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da6a9e-bdb9-4795-8352-1d857b7671d1 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:26.807161 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05635db0-e06a-422c-adb4-1f29578f7291 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:26.821175 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b925d860-5c88-4f6d-a0f9-5baf90cebeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:26.822636 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925d860-5c88-4f6d-a0f9-5baf90cebeec None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:26.823365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925d860-5c88-4f6d-a0f9-5baf90cebeec None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:26.823856 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925d860-5c88-4f6d-a0f9-5baf90cebeec None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:26.825051 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b925d860-5c88-4f6d-a0f9-5baf90cebeec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:26.825544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b925d860-5c88-4f6d-a0f9-5baf90cebeec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:26.827997 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b925d860-5c88-4f6d-a0f9-5baf90cebeec None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:26.843460 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:26.850534 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a98da8-4afc-4fbb-a0dc-c4faf3e4737b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:26.851021 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 463/906] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:26.855707 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a98da8-4afc-4fbb-a0dc-c4faf3e4737b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:26.855707 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a98da8-4afc-4fbb-a0dc-c4faf3e4737b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:26.855707 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a98da8-4afc-4fbb-a0dc-c4faf3e4737b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:26.855707 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a98da8-4afc-4fbb-a0dc-c4faf3e4737b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:26.855707 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a98da8-4afc-4fbb-a0dc-c4faf3e4737b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:26.856385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b925d860-5c88-4f6d-a0f9-5baf90cebeec None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:26.872802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d9a98da8-4afc-4fbb-a0dc-c4faf3e4737b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 04:47:26.872802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:26.872802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 04:47:26.872802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 04:47:26.872802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 04:47:26.872802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 04:47:26.872802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 04:47:26.872802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:26.872802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 04:47:26.872802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:26.872802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:47:26.872802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:26.872802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:26.872802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:26.872802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:26.872802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:26.872802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:26.873876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:26.873876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:26.873876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:26.873876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:26.873876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:26.873876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:26.873876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 04:47:26.873876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 04:47:26.873876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 04:47:26.873876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 04:47:26.873876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:26.873876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:26.873876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:26.873876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:26.873876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:26.873876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:26.874704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:26.874704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:26.874704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:26.874704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:26.874704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:26.874704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:26.874704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:26.874704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:26.874704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:26.874704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:26.874704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 04:47:26.874704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:26.874704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 04:47:26.874704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:26.877736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 444/907] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jul 26 04:47:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:47:26.884147 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a98da8-4afc-4fbb-a0dc-c4faf3e4737b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:27.047769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a98da8-4afc-4fbb-a0dc-c4faf3e4737b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:27.067304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b925d860-5c88-4f6d-a0f9-5baf90cebeec None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:27.127898 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7be103-79de-4e14-a280-50754addfc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:27.128832 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7be103-79de-4e14-a280-50754addfc25 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:27.130033 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7be103-79de-4e14-a280-50754addfc25 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:27.131143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7be103-79de-4e14-a280-50754addfc25 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:27.132039 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7be103-79de-4e14-a280-50754addfc25 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:27.132651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7be103-79de-4e14-a280-50754addfc25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:27.135506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7be103-79de-4e14-a280-50754addfc25 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:27.144343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e863a88-c7b7-44cb-b734-02b449402118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:27.147043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e863a88-c7b7-44cb-b734-02b449402118 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:27.147569 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e863a88-c7b7-44cb-b734-02b449402118 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:27.148304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e863a88-c7b7-44cb-b734-02b449402118 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:27.148480 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 445/908] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jul 26 04:47:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:27.152554 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7be103-79de-4e14-a280-50754addfc25 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:27.167175 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e863a88-c7b7-44cb-b734-02b449402118 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:27.169111 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e863a88-c7b7-44cb-b734-02b449402118 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:27.187681 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-9e863a88-c7b7-44cb-b734-02b449402118 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 04:47:27.187681 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:27.187681 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:27.187681 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:27.187681 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:27.187681 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:27.187681 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:27.187681 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:27.187681 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:27.187681 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:27.187681 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:27.187681 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:27.187681 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:47:27.187681 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:47:27.187681 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:47:27.187681 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:47:27.188870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:27.188870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:27.188870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:47:27.188870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:27.188870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:47:27.188870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:27.188870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:27.188870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:27.188870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:27.188870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:27.188870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:27.188870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:27.188870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:27.188870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:27.188870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:27.188870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:27.188870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:27.188870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:27.189830 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:27.189830 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:27.189830 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:47:27.189830 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:47:27.189830 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:47:27.189830 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:47:27.189830 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:47:27.189830 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:47:27.189830 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 04:47:27.189830 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:27.208758 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 464/909] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Jul 26 04:47:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:47:27.210764 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e863a88-c7b7-44cb-b734-02b449402118 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:27.226355 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:27.313661 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:27.345322 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e863a88-c7b7-44cb-b734-02b449402118 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:27.414101 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7be103-79de-4e14-a280-50754addfc25 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:27.423941 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d38a85-4983-43dd-bb78-effbd1bb68c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:27.424580 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d38a85-4983-43dd-bb78-effbd1bb68c4 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:27.424855 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d38a85-4983-43dd-bb78-effbd1bb68c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:27.425122 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d38a85-4983-43dd-bb78-effbd1bb68c4 None admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:27.425930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d38a85-4983-43dd-bb78-effbd1bb68c4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:27.426234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d38a85-4983-43dd-bb78-effbd1bb68c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:27.429719 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d38a85-4983-43dd-bb78-effbd1bb68c4 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:27.447568 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 465/910] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Jul 26 04:47:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:27.453053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d38a85-4983-43dd-bb78-effbd1bb68c4 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:27.467094 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed4181-66d0-45b4-98b2-c25d2af942b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:27.468129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed4181-66d0-45b4-98b2-c25d2af942b2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:27.468612 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed4181-66d0-45b4-98b2-c25d2af942b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:27.469077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed4181-66d0-45b4-98b2-c25d2af942b2 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:27.469984 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed4181-66d0-45b4-98b2-c25d2af942b2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:27.470611 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed4181-66d0-45b4-98b2-c25d2af942b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:27.472147 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed4181-66d0-45b4-98b2-c25d2af942b2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:27.479000 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 446/911] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jul 26 04:47:27 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:27.485875 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed4181-66d0-45b4-98b2-c25d2af942b2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:27.543965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:27.583863 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d38a85-4983-43dd-bb78-effbd1bb68c4 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:27.615036 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed4181-66d0-45b4-98b2-c25d2af942b2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:27.623228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06c2a2-a099-4921-b1fe-6343efa7df8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:27.623896 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06c2a2-a099-4921-b1fe-6343efa7df8f None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:27.624177 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06c2a2-a099-4921-b1fe-6343efa7df8f None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:27.624551 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06c2a2-a099-4921-b1fe-6343efa7df8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:27.644177 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:27.659643 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fa1ef-4ec7-4e33-a7ca-e08d6f84d09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:27.660523 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fa1ef-4ec7-4e33-a7ca-e08d6f84d09a None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:27.660864 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fa1ef-4ec7-4e33-a7ca-e08d6f84d09a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:27.661450 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fa1ef-4ec7-4e33-a7ca-e08d6f84d09a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:27.662444 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fa1ef-4ec7-4e33-a7ca-e08d6f84d09a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:27.663851 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fa1ef-4ec7-4e33-a7ca-e08d6f84d09a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:27.695683 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 447/912] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 04:47:27 2022] POST /identity/v3/endpoints => generated 346 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:47:27.710455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a06c2a2-a099-4921-b1fe-6343efa7df8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:27.712264 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a06c2a2-a099-4921-b1fe-6343efa7df8f None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:27.794806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:27.847371 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 466/913] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:47:27 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 04:47:27.856785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fa1ef-4ec7-4e33-a7ca-e08d6f84d09a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:27.969855 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fa1ef-4ec7-4e33-a7ca-e08d6f84d09a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:28.005965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dc6c9c-36dd-4658-8be9-bdbadd5236e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:28.007048 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc6c9c-36dd-4658-8be9-bdbadd5236e4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:28.007533 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc6c9c-36dd-4658-8be9-bdbadd5236e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:28.008215 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc6c9c-36dd-4658-8be9-bdbadd5236e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:28.021637 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dc6c9c-36dd-4658-8be9-bdbadd5236e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:28.023458 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dc6c9c-36dd-4658-8be9-bdbadd5236e4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:28.038797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-48dc6c9c-36dd-4658-8be9-bdbadd5236e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:28.038797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:28.038797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:28.038797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:28.038797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:28.038797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:28.038797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:28.038797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:28.038797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:28.038797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:28.038797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:28.038797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:28.038797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:28.038797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:28.038797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:28.039769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:28.039769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:28.039769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:28.039769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:28.039769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:28.039769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:28.039769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:28.039769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:28.039769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:28.039769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:28.039769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:28.039769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:28.039769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:28.039769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:28.039769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:28.039769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:28.039769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:28.040806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:28.040806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:28.040806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:28.040806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:28.046209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 448/914] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:28.052799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06c2a2-a099-4921-b1fe-6343efa7df8f None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:28.097309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0b9b9-bd09-4697-b809-2066dbba009d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:28.097893 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0b9b9-bd09-4697-b809-2066dbba009d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:28.097893 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0b9b9-bd09-4697-b809-2066dbba009d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:28.099580 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 449/915] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:47:28.105354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770b7b2-ef7d-4e81-bb54-197f11ceffde None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:28.105673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770b7b2-ef7d-4e81-bb54-197f11ceffde None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:28.105915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770b7b2-ef7d-4e81-bb54-197f11ceffde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:28.130254 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f770b7b2-ef7d-4e81-bb54-197f11ceffde None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:28.131532 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f770b7b2-ef7d-4e81-bb54-197f11ceffde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:28.154502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:28.165131 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06c2a2-a099-4921-b1fe-6343efa7df8f None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:28.210619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4407ecc0-222b-46da-93f2-6b7b45c7821c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:28.211517 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407ecc0-222b-46da-93f2-6b7b45c7821c None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:28.212253 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407ecc0-222b-46da-93f2-6b7b45c7821c None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:28.212616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407ecc0-222b-46da-93f2-6b7b45c7821c None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:28.213637 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4407ecc0-222b-46da-93f2-6b7b45c7821c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:28.213948 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4407ecc0-222b-46da-93f2-6b7b45c7821c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:28.216076 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4407ecc0-222b-46da-93f2-6b7b45c7821c None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:28.223683 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 450/916] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 04:47:28.229181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 467/917] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:28.234698 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770b7b2-ef7d-4e81-bb54-197f11ceffde None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:28.244356 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4407ecc0-222b-46da-93f2-6b7b45c7821c None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:28.275223 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:28.345698 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770b7b2-ef7d-4e81-bb54-197f11ceffde None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:28.365533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4407ecc0-222b-46da-93f2-6b7b45c7821c None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:28.383531 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7927eb4-04be-4701-a251-a1279bdcba37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:28.384226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7927eb4-04be-4701-a251-a1279bdcba37 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:28.384546 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7927eb4-04be-4701-a251-a1279bdcba37 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:28.384825 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7927eb4-04be-4701-a251-a1279bdcba37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:28.397663 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7927eb4-04be-4701-a251-a1279bdcba37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:28.399735 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7927eb4-04be-4701-a251-a1279bdcba37 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:28.410065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9cde9d-c0d6-4a3e-ac79-4ff40183d47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:28.410728 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9cde9d-c0d6-4a3e-ac79-4ff40183d47a None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:28.411015 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9cde9d-c0d6-4a3e-ac79-4ff40183d47a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:28.411294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9cde9d-c0d6-4a3e-ac79-4ff40183d47a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:28.412006 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9cde9d-c0d6-4a3e-ac79-4ff40183d47a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:28.413473 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9cde9d-c0d6-4a3e-ac79-4ff40183d47a None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:28.415770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b7927eb4-04be-4701-a251-a1279bdcba37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:28.415770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:28.415770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:28.415770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:28.415770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:28.415770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:28.415770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:28.415770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:28.415770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:28.415770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:28.415770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:28.415770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:28.415770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:47:28.415770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:47:28.415770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:47:28.416651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:47:28.416651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:28.416651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:28.416651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:28.416651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:28.416651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:28.416651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:28.416651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:28.416651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:28.416651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:28.416651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:28.416651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:28.416651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:28.416651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:28.416651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:28.416651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:28.416651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:28.417531 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:47:28.417531 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:47:28.417531 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:47:28.417531 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:47:28.417531 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:47:28.417531 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:47:28.417531 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:28.417531 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:28.420060 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 451/918] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:47:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:28.423589 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-4e9cde9d-c0d6-4a3e-ac79-4ff40183d47a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 04:47:28.423589 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:28.423589 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 04:47:28.423589 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 04:47:28.423589 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 04:47:28.423589 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 04:47:28.423589 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 04:47:28.423589 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:28.423589 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 04:47:28.423589 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:28.423589 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:47:28.423589 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:28.423589 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:28.423589 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:28.423589 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:28.423589 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:28.424529 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:28.424529 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:28.424529 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:28.424529 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:28.424529 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:28.424529 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:28.424529 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:28.424529 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 04:47:28.424529 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 04:47:28.424529 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 04:47:28.424529 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 04:47:28.424529 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:28.424529 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:28.424529 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:28.424529 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:28.424529 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:28.425345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:28.425345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:28.425345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:28.425345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:28.425345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:28.425345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:28.425345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:28.425345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:28.425345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:28.425345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:28.425345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:28.425345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 04:47:28.425345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:28.425345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 04:47:28.425345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:28.427663 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7927eb4-04be-4701-a251-a1279bdcba37 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:28.430142 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 468/919] 10.222.0.17 () {56 vars in 1132 bytes} [Tue Jul 26 04:47:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:47:28.431195 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9cde9d-c0d6-4a3e-ac79-4ff40183d47a None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:28.449783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:28.532788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:28.546537 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9cde9d-c0d6-4a3e-ac79-4ff40183d47a None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:28.564799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7927eb4-04be-4701-a251-a1279bdcba37 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:28.582733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a4778-7074-458e-9fce-b30ca749fb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:28.583528 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a4778-7074-458e-9fce-b30ca749fb7a None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:28.583947 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a4778-7074-458e-9fce-b30ca749fb7a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:28.584316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a4778-7074-458e-9fce-b30ca749fb7a None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:28.585174 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a4778-7074-458e-9fce-b30ca749fb7a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:28.585577 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a4778-7074-458e-9fce-b30ca749fb7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:28.587077 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a4778-7074-458e-9fce-b30ca749fb7a None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:28.597549 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 469/920] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Jul 26 04:47:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:28.598688 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a4778-7074-458e-9fce-b30ca749fb7a None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:28.610471 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da261a3-a083-4177-b695-e78c7b42b314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:28.611627 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da261a3-a083-4177-b695-e78c7b42b314 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:28.612162 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da261a3-a083-4177-b695-e78c7b42b314 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:28.612639 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da261a3-a083-4177-b695-e78c7b42b314 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:28.613677 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da261a3-a083-4177-b695-e78c7b42b314 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:28.614228 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da261a3-a083-4177-b695-e78c7b42b314 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:28.616146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da261a3-a083-4177-b695-e78c7b42b314 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:28.632698 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 452/921] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:47:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:28.638574 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da261a3-a083-4177-b695-e78c7b42b314 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:28.709879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a4778-7074-458e-9fce-b30ca749fb7a None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:28.739962 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:28.745469 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1050c84f-6261-417c-bf33-57aac7f41a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:28.746070 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050c84f-6261-417c-bf33-57aac7f41a3b None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:28.746410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050c84f-6261-417c-bf33-57aac7f41a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:28.746690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050c84f-6261-417c-bf33-57aac7f41a3b None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:28.747411 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050c84f-6261-417c-bf33-57aac7f41a3b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:28.747714 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050c84f-6261-417c-bf33-57aac7f41a3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:28.749128 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050c84f-6261-417c-bf33-57aac7f41a3b None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:28.755451 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 470/922] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Jul 26 04:47:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:28.756917 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ffa47-d7d4-48b1-8415-8a04ef9f0127 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:28.757207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ffa47-d7d4-48b1-8415-8a04ef9f0127 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:28.757477 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ffa47-d7d4-48b1-8415-8a04ef9f0127 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:28.762838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 471/923] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:28.762838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ffa47-d7d4-48b1-8415-8a04ef9f0127 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:28.774548 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da261a3-a083-4177-b695-e78c7b42b314 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:28.787429 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:28.820609 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afdbbac-4241-4c14-ab3a-1710645c1c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:28.821367 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afdbbac-4241-4c14-ab3a-1710645c1c08 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:28.821655 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afdbbac-4241-4c14-ab3a-1710645c1c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:28.822106 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afdbbac-4241-4c14-ab3a-1710645c1c08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:28.822931 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afdbbac-4241-4c14-ab3a-1710645c1c08 None admin] RBAC: Adding query filter params (role.id=29516d985a494f4b88d90e1e4f525c24, scope.project.id=d583e0be8ef74766bffd6c3c07af18f1, user.id=3a8f3eb811ae40ad9cb64261f95a992e) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:28.823250 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afdbbac-4241-4c14-ab3a-1710645c1c08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:28.824989 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afdbbac-4241-4c14-ab3a-1710645c1c08 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:28.835933 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 453/924] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:47:28 2022] GET /identity/v3/role_assignments?user.id=3a8f3eb811ae40ad9cb64261f95a992e&scope.project.id=d583e0be8ef74766bffd6c3c07af18f1&role.id=29516d985a494f4b88d90e1e4f525c24 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:28.841982 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c8fa6-5e70-4196-ad67-51d2199e5938 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:28.842294 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c8fa6-5e70-4196-ad67-51d2199e5938 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:28.842543 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c8fa6-5e70-4196-ad67-51d2199e5938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:28.866875 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-533c8fa6-5e70-4196-ad67-51d2199e5938 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:28.868087 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-533c8fa6-5e70-4196-ad67-51d2199e5938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:28.896248 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ffa47-d7d4-48b1-8415-8a04ef9f0127 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:28.939735 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d73bf-7489-4f8a-88b6-3c637b15026a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:28.940789 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d73bf-7489-4f8a-88b6-3c637b15026a None admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:28.941237 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d73bf-7489-4f8a-88b6-3c637b15026a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:28.941654 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d73bf-7489-4f8a-88b6-3c637b15026a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:28.942719 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18d73bf-7489-4f8a-88b6-3c637b15026a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:28.944459 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18d73bf-7489-4f8a-88b6-3c637b15026a None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:28.948672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 454/925] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:28.971636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c8fa6-5e70-4196-ad67-51d2199e5938 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:28.986162 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 472/926] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 04:47:28 2022] POST /identity/v3/endpoints => generated 347 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:47:29.058774 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:29.094828 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c8fa6-5e70-4196-ad67-51d2199e5938 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:29.140139 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f0eb1-3ce8-43e8-9df7-e0c5a649b9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:29.140791 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f0eb1-3ce8-43e8-9df7-e0c5a649b9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:29.141140 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f0eb1-3ce8-43e8-9df7-e0c5a649b9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:29.141423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f0eb1-3ce8-43e8-9df7-e0c5a649b9b6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:29.142135 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f0eb1-3ce8-43e8-9df7-e0c5a649b9b6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:29.143612 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f0eb1-3ce8-43e8-9df7-e0c5a649b9b6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:29.153514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-690f0eb1-3ce8-43e8-9df7-e0c5a649b9b6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 04:47:29.153514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:29.153514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 04:47:29.153514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 04:47:29.153514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 04:47:29.153514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 04:47:29.153514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 04:47:29.153514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:29.153514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 04:47:29.153514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:29.153514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:47:29.153514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:29.153514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:29.153514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:29.153514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:29.153514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:29.153514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:29.154591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:29.154591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:29.154591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:29.154591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:29.154591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:29.154591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:29.154591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 04:47:29.154591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 04:47:29.154591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 04:47:29.154591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 04:47:29.154591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:29.154591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:29.154591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:29.154591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:29.154591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:29.154591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:29.155437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:29.155437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:29.155437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:29.155437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:29.155437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:29.155437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:29.155437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:29.155437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:29.155437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:29.155437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:29.155437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 04:47:29.155437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:29.155437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 04:47:29.155437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:29.155437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 455/927] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Jul 26 04:47:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:47:29.162255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d73bf-7489-4f8a-88b6-3c637b15026a None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:29.248127 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:29.283738 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d73bf-7489-4f8a-88b6-3c637b15026a None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:29.321403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3fb1fb-14b1-45d1-9270-b18d9af85c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:29.322462 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fb1fb-14b1-45d1-9270-b18d9af85c6b None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:29.322976 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fb1fb-14b1-45d1-9270-b18d9af85c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:29.323488 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fb1fb-14b1-45d1-9270-b18d9af85c6b None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:29.324444 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3fb1fb-14b1-45d1-9270-b18d9af85c6b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:29.324973 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3fb1fb-14b1-45d1-9270-b18d9af85c6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:29.326688 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3fb1fb-14b1-45d1-9270-b18d9af85c6b None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:29.332499 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 473/928] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Jul 26 04:47:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:29.343701 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f0eb1-3ce8-43e8-9df7-e0c5a649b9b6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:29.450367 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f0eb1-3ce8-43e8-9df7-e0c5a649b9b6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:29.480182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:29.488263 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a22ae0-b9d9-4a9e-89ed-894eeb5e7988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:29.488916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a22ae0-b9d9-4a9e-89ed-894eeb5e7988 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:29.489204 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a22ae0-b9d9-4a9e-89ed-894eeb5e7988 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:29.489486 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a22ae0-b9d9-4a9e-89ed-894eeb5e7988 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:29.490365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a22ae0-b9d9-4a9e-89ed-894eeb5e7988 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:29.490676 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a22ae0-b9d9-4a9e-89ed-894eeb5e7988 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:29.492129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a22ae0-b9d9-4a9e-89ed-894eeb5e7988 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:29.498357 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 456/929] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Jul 26 04:47:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:29.504316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3fb1fb-14b1-45d1-9270-b18d9af85c6b None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:29.615313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3fb1fb-14b1-45d1-9270-b18d9af85c6b None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:29.656116 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761cce5-4976-484e-83a1-3f52a4668028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:29.656676 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761cce5-4976-484e-83a1-3f52a4668028 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:29.656962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761cce5-4976-484e-83a1-3f52a4668028 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:29.657237 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761cce5-4976-484e-83a1-3f52a4668028 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:29.658005 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761cce5-4976-484e-83a1-3f52a4668028 None admin] RBAC: Adding query filter params (interface=public, service_id=7f16dcb1cca742ddaefab6bf1923efeb, region_id=RegionOne) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:29.658315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761cce5-4976-484e-83a1-3f52a4668028 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:29.659734 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761cce5-4976-484e-83a1-3f52a4668028 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:29.662315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:29.667592 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 474/930] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Jul 26 04:47:29 2022] GET /identity/v3/endpoints?service_id=7f16dcb1cca742ddaefab6bf1923efeb&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:29.673143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a22ae0-b9d9-4a9e-89ed-894eeb5e7988 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:29.783801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a22ae0-b9d9-4a9e-89ed-894eeb5e7988 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:29.818659 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f35d0-fb4b-4a4a-ba89-53db848ea517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:29.819705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f35d0-fb4b-4a4a-ba89-53db848ea517 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:29.820192 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f35d0-fb4b-4a4a-ba89-53db848ea517 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:29.820665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f35d0-fb4b-4a4a-ba89-53db848ea517 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:29.821547 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750f35d0-fb4b-4a4a-ba89-53db848ea517 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:29.823271 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750f35d0-fb4b-4a4a-ba89-53db848ea517 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:29.830202 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 457/931] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Jul 26 04:47:29 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 04:47:29.847582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56285dc1-47bb-4e5d-b116-b1c5e20c9844 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:29.847909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56285dc1-47bb-4e5d-b116-b1c5e20c9844 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:29.848140 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56285dc1-47bb-4e5d-b116-b1c5e20c9844 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:29.849856 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 475/932] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:29.855821 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e06964-032c-4ab0-bcb5-17dcb98644e1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:29.856442 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e06964-032c-4ab0-bcb5-17dcb98644e1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:29.856927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e06964-032c-4ab0-bcb5-17dcb98644e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:29.877333 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-47e06964-032c-4ab0-bcb5-17dcb98644e1 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:29.878575 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-47e06964-032c-4ab0-bcb5-17dcb98644e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:29.941358 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:29.957725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 458/933] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:29.982653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56285dc1-47bb-4e5d-b116-b1c5e20c9844 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:30.112787 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56285dc1-47bb-4e5d-b116-b1c5e20c9844 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:30.148417 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48847438-6335-4c84-b39e-98902afbfc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:30.149042 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48847438-6335-4c84-b39e-98902afbfc7e None admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:30.149324 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48847438-6335-4c84-b39e-98902afbfc7e None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:30.149598 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48847438-6335-4c84-b39e-98902afbfc7e None admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:30.150358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48847438-6335-4c84-b39e-98902afbfc7e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:30.151835 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48847438-6335-4c84-b39e-98902afbfc7e None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:30.154486 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: WARNING keystone.server.flask.common [None req-48847438-6335-4c84-b39e-98902afbfc7e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 26 04:47:30.174044 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.sql.core [None req-48847438-6335-4c84-b39e-98902afbfc7e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 26 04:47:30.174044 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 26 04:47:30.174044 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [parameters: {'id': '871da3d7d04c410680b2d71bc6aff674', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 26 04:47:30.174044 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80205) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 26 04:47:30.191516 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-48847438-6335-4c84-b39e-98902afbfc7e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 871da3d7d04c410680b2d71bc6aff674.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 871da3d7d04c410680b2d71bc6aff674. Jul 26 04:47:30.191516 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:30.191516 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 04:47:30.191516 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 04:47:30.191516 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 04:47:30.191516 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 04:47:30.191516 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 04:47:30.191516 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 04:47:30.191516 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 04:47:30.191516 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application conn.query(q) Jul 26 04:47:30.191516 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 04:47:30.191516 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 04:47:30.191516 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 04:47:30.192403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application result.read() Jul 26 04:47:30.192403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 04:47:30.192403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 04:47:30.192403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 04:47:30.192403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 04:47:30.192403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 04:47:30.192403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 04:47:30.192403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 04:47:30.192403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 04:47:30.192403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 26 04:47:30.192403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:30.192403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 26 04:47:30.192403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:30.192403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:30.192403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 26 04:47:30.192403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 26 04:47:30.192403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 26 04:47:30.192403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 26 04:47:30.193386 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 04:47:30.193386 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application next(self.gen) Jul 26 04:47:30.193386 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 26 04:47:30.193386 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application yield resource Jul 26 04:47:30.193386 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 04:47:30.193386 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application next(self.gen) Jul 26 04:47:30.193386 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 26 04:47:30.193386 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self.session.rollback() Jul 26 04:47:30.193386 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 26 04:47:30.193386 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self.force_reraise() Jul 26 04:47:30.193386 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 26 04:47:30.193386 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise self.value Jul 26 04:47:30.193386 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 26 04:47:30.193386 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 26 04:47:30.193386 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 26 04:47:30.193386 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application session.commit() Jul 26 04:47:30.193386 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 26 04:47:30.195362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 26 04:47:30.195362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 26 04:47:30.195362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self._prepare_impl() Jul 26 04:47:30.195362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 26 04:47:30.195362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self.session.flush() Jul 26 04:47:30.195362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 26 04:47:30.195362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self._flush(objects) Jul 26 04:47:30.195362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 26 04:47:30.195362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 26 04:47:30.195362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 26 04:47:30.195362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application compat.raise_( Jul 26 04:47:30.195362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 04:47:30.195362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception Jul 26 04:47:30.195362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 26 04:47:30.195362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application flush_context.execute() Jul 26 04:47:30.195362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 26 04:47:30.195362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rec.execute(self) Jul 26 04:47:30.197645 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 26 04:47:30.197645 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 26 04:47:30.197645 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 26 04:47:30.197645 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 26 04:47:30.197645 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 26 04:47:30.197645 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 26 04:47:30.197645 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 26 04:47:30.197645 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 26 04:47:30.197645 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 26 04:47:30.197645 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 26 04:47:30.197645 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 26 04:47:30.197645 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 26 04:47:30.197645 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 26 04:47:30.197645 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 26 04:47:30.197645 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 26 04:47:30.199848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 26 04:47:30.199848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 04:47:30.199848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception Jul 26 04:47:30.199848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 04:47:30.199848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 04:47:30.199848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 04:47:30.199848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 04:47:30.199848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 04:47:30.199848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 04:47:30.199848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 04:47:30.199848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application conn.query(q) Jul 26 04:47:30.199848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 04:47:30.199848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 04:47:30.199848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 04:47:30.199848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application result.read() Jul 26 04:47:30.199848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 04:47:30.199848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 04:47:30.201936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 04:47:30.201936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 04:47:30.201936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 04:47:30.201936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 04:47:30.201936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 04:47:30.201936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 04:47:30.201936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 26 04:47:30.201936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 26 04:47:30.201936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [parameters: {'id': '871da3d7d04c410680b2d71bc6aff674', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 26 04:47:30.201936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 26 04:47:30.201936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:30.201936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:47:30.201936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:30.201936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:30.201936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 26 04:47:30.208422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 26 04:47:30.208422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 26 04:47:30.208422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 26 04:47:30.208422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 26 04:47:30.208422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:30.208422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:47:30.208422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:30.208422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:30.208422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:30.208422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:30.208422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:30.208422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:30.208422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:30.208422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:30.208422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:30.208422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:30.208422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:30.210563 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:30.210563 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 26 04:47:30.210563 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 26 04:47:30.210563 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:30.210563 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:30.210563 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 26 04:47:30.210563 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 26 04:47:30.210563 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 871da3d7d04c410680b2d71bc6aff674. Jul 26 04:47:30.210563 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:30.210563 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 476/934] 10.222.0.17 () {60 vars in 1140 bytes} [Tue Jul 26 04:47:29 2022] POST /identity/v3/projects => generated 262 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:30.211867 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e06964-032c-4ab0-bcb5-17dcb98644e1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:30.310056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e06964-032c-4ab0-bcb5-17dcb98644e1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:30.346639 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534873e-2f7e-4c16-8d1b-b748911e7bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:30.347306 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534873e-2f7e-4c16-8d1b-b748911e7bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:30.347611 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534873e-2f7e-4c16-8d1b-b748911e7bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:30.347891 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534873e-2f7e-4c16-8d1b-b748911e7bc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:30.356105 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534873e-2f7e-4c16-8d1b-b748911e7bc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:30.357691 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534873e-2f7e-4c16-8d1b-b748911e7bc4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:30.369477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-6534873e-2f7e-4c16-8d1b-b748911e7bc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:30.369477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:30.369477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:30.369477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:30.369477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:30.369477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:30.369477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:30.369477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:30.369477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:30.369477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:30.369477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:30.369477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:30.369477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:47:30.369477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:47:30.369477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:47:30.371543 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:47:30.371543 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:30.371543 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:30.371543 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:30.371543 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:30.371543 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:30.371543 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:30.371543 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:30.371543 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:30.371543 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:30.371543 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:30.371543 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:30.371543 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:30.371543 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:30.371543 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:30.371543 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:30.371543 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:30.373813 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:47:30.373813 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:47:30.373813 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:47:30.373813 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:47:30.373813 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:47:30.373813 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:47:30.373813 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:30.373813 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:30.373813 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 459/935] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:47:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:30.377255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48847438-6335-4c84-b39e-98902afbfc7e None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:30.445688 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:30.494208 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48847438-6335-4c84-b39e-98902afbfc7e None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:30.530007 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b879089d-b6f8-4fde-b3a2-554783771545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:30.530638 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879089d-b6f8-4fde-b3a2-554783771545 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:30.530921 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879089d-b6f8-4fde-b3a2-554783771545 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:30.531192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879089d-b6f8-4fde-b3a2-554783771545 None admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:30.531957 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b879089d-b6f8-4fde-b3a2-554783771545 None admin] RBAC: Adding query filter params (name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:30.532260 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b879089d-b6f8-4fde-b3a2-554783771545 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:30.533923 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b879089d-b6f8-4fde-b3a2-554783771545 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:30.555177 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 477/936] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Jul 26 04:47:30 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:30.958025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be83ad0-252f-409b-b65c-dcdd6739dc27 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:30.958379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be83ad0-252f-409b-b65c-dcdd6739dc27 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:30.958591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be83ad0-252f-409b-b65c-dcdd6739dc27 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:30.960547 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 460/937] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:30.966779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901c4f3-a732-4fca-ba0f-4e1e80c99767 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:30.967072 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901c4f3-a732-4fca-ba0f-4e1e80c99767 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:30.967384 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901c4f3-a732-4fca-ba0f-4e1e80c99767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:30.986724 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-a901c4f3-a732-4fca-ba0f-4e1e80c99767 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:30.988292 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a901c4f3-a732-4fca-ba0f-4e1e80c99767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:31.065266 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 478/938] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:31.087532 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be83ad0-252f-409b-b65c-dcdd6739dc27 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:31.146090 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:31.196048 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be83ad0-252f-409b-b65c-dcdd6739dc27 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:31.232853 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e5fa0-f602-4906-9830-005a2a64e7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:31.233483 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e5fa0-f602-4906-9830-005a2a64e7de None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:31.233702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e5fa0-f602-4906-9830-005a2a64e7de None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:31.233979 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e5fa0-f602-4906-9830-005a2a64e7de None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:31.238085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6966ae7-9ce5-4dc3-bd6f-7040a1249d99 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:31.238377 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6966ae7-9ce5-4dc3-bd6f-7040a1249d99 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:31.238655 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6966ae7-9ce5-4dc3-bd6f-7040a1249d99 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:31.240401 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 479/939] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:47:31.245938 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1bb32-0f82-4717-ae1f-7dc091dfa528 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:31.246074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5e5fa0-f602-4906-9830-005a2a64e7de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:47:31.246300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1bb32-0f82-4717-ae1f-7dc091dfa528 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:31.246572 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1bb32-0f82-4717-ae1f-7dc091dfa528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:31.251143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-ea5e5fa0-f602-4906-9830-005a2a64e7de None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 04:47:31.251143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:31.251143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:31.251143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:31.251143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:31.251143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:31.251143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:31.251143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:31.251143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:31.251143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:31.251143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:31.251143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:31.251143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:47:31.251143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:47:31.251143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:47:31.251143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:47:31.252097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:47:31.252097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:47:31.252097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:47:31.252097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:47:31.252097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:31.252097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:31.252097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:31.252097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:31.252097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:31.252097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:31.252097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:31.252097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:31.252097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:31.252097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:31.252097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:31.252097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:31.252930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:31.252930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:31.252930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:31.252930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:31.252930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:47:31.252930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:47:31.252930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:47:31.252930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:47:31.252930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:47:31.252930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:47:31.252930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 04:47:31.252930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:31.257456 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 461/940] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Jul 26 04:47:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:31.258720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e5fa0-f602-4906-9830-005a2a64e7de None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:31.266253 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-a8a1bb32-0f82-4717-ae1f-7dc091dfa528 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:31.267529 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a8a1bb32-0f82-4717-ae1f-7dc091dfa528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:31.315197 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:31.351861 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 480/941] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 04:47:31.368528 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e5fa0-f602-4906-9830-005a2a64e7de None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:31.368948 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1bb32-0f82-4717-ae1f-7dc091dfa528 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:31.379625 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:31.408224 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c7697-4376-4f8a-9e29-e70f0afcaa0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:31.408919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c7697-4376-4f8a-9e29-e70f0afcaa0d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:31.409208 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c7697-4376-4f8a-9e29-e70f0afcaa0d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:31.409504 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c7697-4376-4f8a-9e29-e70f0afcaa0d None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:31.410267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c7697-4376-4f8a-9e29-e70f0afcaa0d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:31.410603 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c7697-4376-4f8a-9e29-e70f0afcaa0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:31.412885 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c7697-4376-4f8a-9e29-e70f0afcaa0d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:31.426626 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 462/942] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Jul 26 04:47:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:31.428145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c7697-4376-4f8a-9e29-e70f0afcaa0d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:31.483898 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1bb32-0f82-4717-ae1f-7dc091dfa528 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:31.522931 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce407ca4-daf2-4c66-aa0b-95cd5dd0004d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:31.523986 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce407ca4-daf2-4c66-aa0b-95cd5dd0004d None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:31.524491 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce407ca4-daf2-4c66-aa0b-95cd5dd0004d None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:31.525019 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce407ca4-daf2-4c66-aa0b-95cd5dd0004d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:31.525979 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce407ca4-daf2-4c66-aa0b-95cd5dd0004d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:31.527685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce407ca4-daf2-4c66-aa0b-95cd5dd0004d None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:31.537718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-ce407ca4-daf2-4c66-aa0b-95cd5dd0004d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 04:47:31.537718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:31.537718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 04:47:31.537718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 04:47:31.537718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 04:47:31.537718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 04:47:31.537718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 04:47:31.537718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:31.537718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 04:47:31.537718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:31.537718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:47:31.537718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:31.537718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:31.537718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:31.537718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:31.537718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:31.537718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:31.538753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:31.538753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:31.538753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:31.538753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:31.538753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:31.538753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:31.538753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 04:47:31.538753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 04:47:31.538753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 04:47:31.538753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 04:47:31.538753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:31.538753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:31.538753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:31.538753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:31.538753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:31.538753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:31.539619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:31.539619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:31.539619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:31.539619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:31.539619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:31.539619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:31.539619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:31.539619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:31.539619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:31.539619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:31.539619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 04:47:31.539619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:31.539619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 04:47:31.539619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:31.541898 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c7697-4376-4f8a-9e29-e70f0afcaa0d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:31.547609 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 481/943] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:47:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:31.548812 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce407ca4-daf2-4c66-aa0b-95cd5dd0004d None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:31.579470 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb830541-ca6c-46b5-a6c1-2259994dff11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:31.580440 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb830541-ca6c-46b5-a6c1-2259994dff11 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:31.580621 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb830541-ca6c-46b5-a6c1-2259994dff11 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:31.580867 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb830541-ca6c-46b5-a6c1-2259994dff11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:31.593013 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb830541-ca6c-46b5-a6c1-2259994dff11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:31.594668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb830541-ca6c-46b5-a6c1-2259994dff11 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:31.597149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:31.612221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-fb830541-ca6c-46b5-a6c1-2259994dff11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:31.612221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:31.612221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:31.612221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:31.612221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:31.612221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:31.612221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:31.612221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:31.612221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:31.612221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:31.612221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:31.612221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:31.612221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:31.612221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:31.612221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:31.613201 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:31.613201 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:31.613201 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:31.613201 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:31.613201 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:31.613201 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:31.613201 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:31.613201 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:31.613201 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:31.613201 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:31.613201 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:31.613201 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:31.613201 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:31.613201 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:31.613201 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:31.613201 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:31.613201 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:31.614128 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:31.614128 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:31.614128 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:31.614128 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:31.621059 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 463/944] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:31.622637 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb830541-ca6c-46b5-a6c1-2259994dff11 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:31.649983 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:31.663470 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce407ca4-daf2-4c66-aa0b-95cd5dd0004d None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:31.701369 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf40857-913c-45fb-a6dd-766d76cce94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:31.702152 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf40857-913c-45fb-a6dd-766d76cce94c None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:31.702519 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf40857-913c-45fb-a6dd-766d76cce94c None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:31.702864 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf40857-913c-45fb-a6dd-766d76cce94c None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:31.703704 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf40857-913c-45fb-a6dd-766d76cce94c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:31.704084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf40857-913c-45fb-a6dd-766d76cce94c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:31.705565 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf40857-913c-45fb-a6dd-766d76cce94c None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:31.715824 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 482/945] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Jul 26 04:47:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:31.717563 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf40857-913c-45fb-a6dd-766d76cce94c None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:31.735515 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb830541-ca6c-46b5-a6c1-2259994dff11 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:31.773986 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59a4e6-83e4-4cae-904f-b4f78cc93877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:31.775008 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59a4e6-83e4-4cae-904f-b4f78cc93877 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:31.775505 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59a4e6-83e4-4cae-904f-b4f78cc93877 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:31.775989 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59a4e6-83e4-4cae-904f-b4f78cc93877 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:31.776871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d59a4e6-83e4-4cae-904f-b4f78cc93877 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:31.777356 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d59a4e6-83e4-4cae-904f-b4f78cc93877 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:31.778935 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d59a4e6-83e4-4cae-904f-b4f78cc93877 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:31.795679 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 464/946] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:31.797991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59a4e6-83e4-4cae-904f-b4f78cc93877 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:31.856472 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf40857-913c-45fb-a6dd-766d76cce94c None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:31.859789 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:31.892574 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:31.910481 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e78da-24f1-4146-bb61-7ebb70c023e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:31.911800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e78da-24f1-4146-bb61-7ebb70c023e4 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:31.912265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e78da-24f1-4146-bb61-7ebb70c023e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:31.912748 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e78da-24f1-4146-bb61-7ebb70c023e4 None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:31.913314 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59a4e6-83e4-4cae-904f-b4f78cc93877 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:31.913809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e78da-24f1-4146-bb61-7ebb70c023e4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:31.914305 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e78da-24f1-4146-bb61-7ebb70c023e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:31.917197 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e78da-24f1-4146-bb61-7ebb70c023e4 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:31.928487 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 483/947] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Jul 26 04:47:31 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:31.957616 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3fef35-3c69-4aa5-bf66-9d9e2ca44aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:31.958309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fef35-3c69-4aa5-bf66-9d9e2ca44aec None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:31.958585 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fef35-3c69-4aa5-bf66-9d9e2ca44aec None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:31.958867 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fef35-3c69-4aa5-bf66-9d9e2ca44aec None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:31.965852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19768cd9-66da-4d4b-9858-f295ca5a9569 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:31.966370 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19768cd9-66da-4d4b-9858-f295ca5a9569 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:31.966827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19768cd9-66da-4d4b-9858-f295ca5a9569 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:31.968424 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3fef35-3c69-4aa5-bf66-9d9e2ca44aec None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:31.969388 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 484/948] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:31.969923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3fef35-3c69-4aa5-bf66-9d9e2ca44aec None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:31.976282 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b0028-2e2a-4a42-845b-5efa1a504e87 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:31.976592 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b0028-2e2a-4a42-845b-5efa1a504e87 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:31.976833 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b0028-2e2a-4a42-845b-5efa1a504e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:31.990178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-eb3fef35-3c69-4aa5-bf66-9d9e2ca44aec None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 04:47:31.990178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:31.990178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:31.990178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:31.990178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:31.990178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:31.990178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:31.990178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:31.990178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:31.990178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:31.990178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:31.990178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:31.990178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:47:31.990178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:47:31.990178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:47:31.990178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:47:31.991391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:31.991391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:31.991391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:47:31.991391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:31.991391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:47:31.991391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:31.991391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:31.991391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:31.991391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:31.991391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:31.991391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:31.991391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:31.991391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:31.991391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:31.991391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:31.991391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:31.991391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:31.991391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:31.992587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:31.992587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:31.992587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:47:31.992587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:47:31.992587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:47:31.992587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:47:31.992587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:47:31.992587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:47:31.992587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 04:47:31.992587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:31.995127 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 465/949] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Jul 26 04:47:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:47:32.000759 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3fef35-3c69-4aa5-bf66-9d9e2ca44aec None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:32.001723 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-887b0028-2e2a-4a42-845b-5efa1a504e87 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:32.003004 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-887b0028-2e2a-4a42-845b-5efa1a504e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:32.088249 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 485/950] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:32.110725 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887b0028-2e2a-4a42-845b-5efa1a504e87 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:32.141864 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3fef35-3c69-4aa5-bf66-9d9e2ca44aec None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:32.180282 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef870d7-1e62-4385-89d9-6d45f868b613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:32.181379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef870d7-1e62-4385-89d9-6d45f868b613 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:32.182328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef870d7-1e62-4385-89d9-6d45f868b613 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:32.182797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef870d7-1e62-4385-89d9-6d45f868b613 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:32.184110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef870d7-1e62-4385-89d9-6d45f868b613 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:32.184780 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef870d7-1e62-4385-89d9-6d45f868b613 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:32.187062 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef870d7-1e62-4385-89d9-6d45f868b613 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:32.210356 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:32.210839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 466/951] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Jul 26 04:47:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:32.217165 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef870d7-1e62-4385-89d9-6d45f868b613 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:32.244153 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887b0028-2e2a-4a42-845b-5efa1a504e87 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:32.247954 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:32.289743 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bac3e-761f-4176-b3c7-97bf9fdf644e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:32.290594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bac3e-761f-4176-b3c7-97bf9fdf644e None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:32.291013 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bac3e-761f-4176-b3c7-97bf9fdf644e None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:32.291300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bac3e-761f-4176-b3c7-97bf9fdf644e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:32.292257 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58bac3e-761f-4176-b3c7-97bf9fdf644e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:32.294535 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58bac3e-761f-4176-b3c7-97bf9fdf644e None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:32.310198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-f58bac3e-761f-4176-b3c7-97bf9fdf644e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 04:47:32.310198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:32.310198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 04:47:32.310198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 04:47:32.310198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 04:47:32.310198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 04:47:32.310198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 04:47:32.310198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:32.310198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 04:47:32.310198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:32.310198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:47:32.310198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:32.310198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:32.310198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:32.310198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:32.310198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:32.310198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:32.311255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:32.311255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:32.311255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:32.311255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:32.311255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:32.311255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:32.311255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 04:47:32.311255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 04:47:32.311255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 04:47:32.311255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 04:47:32.311255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:32.311255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:32.311255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:32.311255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:32.311255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:32.311255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:32.312120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:32.312120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:32.312120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:32.312120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:32.312120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:32.312120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:32.312120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:32.312120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:32.312120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:32.312120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:32.312120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 04:47:32.312120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:32.312120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 04:47:32.312120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:32.313768 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 486/952] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Jul 26 04:47:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:32.318662 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bac3e-761f-4176-b3c7-97bf9fdf644e None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:32.353227 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef870d7-1e62-4385-89d9-6d45f868b613 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:32.397179 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46910a2-7a18-4659-bb0f-3dc88527924f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:32.398212 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46910a2-7a18-4659-bb0f-3dc88527924f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:32.398695 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46910a2-7a18-4659-bb0f-3dc88527924f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:32.399167 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46910a2-7a18-4659-bb0f-3dc88527924f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:32.410267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46910a2-7a18-4659-bb0f-3dc88527924f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:32.412079 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46910a2-7a18-4659-bb0f-3dc88527924f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:32.428248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d46910a2-7a18-4659-bb0f-3dc88527924f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:32.428248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:32.428248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:32.428248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:32.428248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:32.428248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:32.428248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:32.428248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:32.428248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:32.428248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:32.428248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:32.428248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:32.428248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:32.428248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:32.428248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:32.429315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:32.429315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:32.429315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:32.429315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:32.429315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:32.429315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:32.429315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:32.429315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:32.429315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:32.429315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:32.429315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:32.429315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:32.429315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:32.429315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:32.429315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:32.429315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:32.429315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:32.430410 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:32.430410 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:32.430410 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:32.430410 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:32.435916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 467/953] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:32.439090 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46910a2-7a18-4659-bb0f-3dc88527924f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:32.456856 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bac3e-761f-4176-b3c7-97bf9fdf644e None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:32.501117 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e7601-a69f-42ab-9c2e-03aa552693e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:32.501765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e7601-a69f-42ab-9c2e-03aa552693e2 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:32.502045 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e7601-a69f-42ab-9c2e-03aa552693e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:32.502397 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e7601-a69f-42ab-9c2e-03aa552693e2 None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:32.503186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e7601-a69f-42ab-9c2e-03aa552693e2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:32.503583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e7601-a69f-42ab-9c2e-03aa552693e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:32.505547 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e7601-a69f-42ab-9c2e-03aa552693e2 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:32.508687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:32.516607 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 487/954] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Jul 26 04:47:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:32.518344 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e7601-a69f-42ab-9c2e-03aa552693e2 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:32.553130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:32.590053 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46910a2-7a18-4659-bb0f-3dc88527924f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:32.633042 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b8748-1eb4-4948-9afc-38fc1580d9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:32.633821 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b8748-1eb4-4948-9afc-38fc1580d9c1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:32.634232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b8748-1eb4-4948-9afc-38fc1580d9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:32.634553 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b8748-1eb4-4948-9afc-38fc1580d9c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:32.635391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b8748-1eb4-4948-9afc-38fc1580d9c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:32.635852 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b8748-1eb4-4948-9afc-38fc1580d9c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:32.637008 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e7601-a69f-42ab-9c2e-03aa552693e2 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:32.637432 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b8748-1eb4-4948-9afc-38fc1580d9c1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:32.656271 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 468/955] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:32.657634 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b8748-1eb4-4948-9afc-38fc1580d9c1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:32.682358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2e757-f962-42a0-8649-22d2c83a215c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:32.683132 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2e757-f962-42a0-8649-22d2c83a215c None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:32.683415 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2e757-f962-42a0-8649-22d2c83a215c None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:32.683960 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2e757-f962-42a0-8649-22d2c83a215c None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:32.685072 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2e757-f962-42a0-8649-22d2c83a215c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:32.685551 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2e757-f962-42a0-8649-22d2c83a215c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:32.687298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2e757-f962-42a0-8649-22d2c83a215c None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:32.695630 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 488/956] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Jul 26 04:47:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:32.697024 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30dff57-fe51-43b9-9cde-57f4ce638aa2 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:32.697313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30dff57-fe51-43b9-9cde-57f4ce638aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:32.697590 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30dff57-fe51-43b9-9cde-57f4ce638aa2 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:32.699023 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 489/957] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:32.700225 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30dff57-fe51-43b9-9cde-57f4ce638aa2 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:32.772711 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b8748-1eb4-4948-9afc-38fc1580d9c1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:32.788856 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:32.803797 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:32.815464 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eccded8-0a87-446a-a978-bab5797d0b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:32.816258 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccded8-0a87-446a-a978-bab5797d0b15 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:32.816547 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccded8-0a87-446a-a978-bab5797d0b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:32.816832 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccded8-0a87-446a-a978-bab5797d0b15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:32.819031 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30dff57-fe51-43b9-9cde-57f4ce638aa2 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:32.824224 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eccded8-0a87-446a-a978-bab5797d0b15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:32.825725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eccded8-0a87-446a-a978-bab5797d0b15 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:32.838506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-8eccded8-0a87-446a-a978-bab5797d0b15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:32.838506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:32.838506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:32.838506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:32.838506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:32.838506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:32.838506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:32.838506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:32.838506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:32.838506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:32.838506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:32.838506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:32.838506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:47:32.838506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:47:32.838506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:47:32.839869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:47:32.839869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:32.839869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:32.839869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:32.839869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:32.839869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:32.839869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:32.839869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:32.839869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:32.839869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:32.839869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:32.839869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:32.839869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:32.839869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:32.839869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:32.839869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:32.839869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:32.840946 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:47:32.840946 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:47:32.840946 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:47:32.840946 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:47:32.840946 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:47:32.840946 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:47:32.840946 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:32.840946 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:32.845220 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 469/958] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:47:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:32.846562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc851da-d2c7-4590-8d73-6b7903c4ffb0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:32.846856 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc851da-d2c7-4590-8d73-6b7903c4ffb0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:32.847137 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc851da-d2c7-4590-8d73-6b7903c4ffb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:32.859897 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97e635-b36b-4508-a188-5d3cba3447bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:32.860783 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97e635-b36b-4508-a188-5d3cba3447bc None admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:32.861351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97e635-b36b-4508-a188-5d3cba3447bc None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:32.861766 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97e635-b36b-4508-a188-5d3cba3447bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:32.862625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97e635-b36b-4508-a188-5d3cba3447bc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:32.864197 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97e635-b36b-4508-a188-5d3cba3447bc None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:32.865354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5cc851da-d2c7-4590-8d73-6b7903c4ffb0 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:32.866567 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5cc851da-d2c7-4590-8d73-6b7903c4ffb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:32.886501 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 490/959] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 04:47:32 2022] POST /identity/v3/endpoints => generated 349 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 04:47:32.887476 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97e635-b36b-4508-a188-5d3cba3447bc None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:32.974898 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 470/960] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:32.998605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc851da-d2c7-4590-8d73-6b7903c4ffb0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:33.017036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97e635-b36b-4508-a188-5d3cba3447bc None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:33.057414 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1b0c2-98ab-455d-9bb8-e179ec5e416d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:33.057972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1b0c2-98ab-455d-9bb8-e179ec5e416d None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:33.058262 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1b0c2-98ab-455d-9bb8-e179ec5e416d None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:33.058542 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1b0c2-98ab-455d-9bb8-e179ec5e416d None admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:33.059328 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db1b0c2-98ab-455d-9bb8-e179ec5e416d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:33.059716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db1b0c2-98ab-455d-9bb8-e179ec5e416d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:33.061202 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db1b0c2-98ab-455d-9bb8-e179ec5e416d None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:33.074613 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 491/961] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:47:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:33.081446 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1b0c2-98ab-455d-9bb8-e179ec5e416d None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:33.091471 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:33.114630 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:33.118911 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc851da-d2c7-4590-8d73-6b7903c4ffb0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:33.164635 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d8b84-3f96-47b1-9dd8-8d14b7dd7b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:33.165591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d8b84-3f96-47b1-9dd8-8d14b7dd7b72 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:33.166186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d8b84-3f96-47b1-9dd8-8d14b7dd7b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:33.166621 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d8b84-3f96-47b1-9dd8-8d14b7dd7b72 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:33.167805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86d8b84-3f96-47b1-9dd8-8d14b7dd7b72 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:33.169773 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86d8b84-3f96-47b1-9dd8-8d14b7dd7b72 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:33.173110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: WARNING keystone.server.flask.common [None req-f86d8b84-3f96-47b1-9dd8-8d14b7dd7b72 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 26 04:47:33.189692 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.sql.core [None req-f86d8b84-3f96-47b1-9dd8-8d14b7dd7b72 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 26 04:47:33.189692 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 26 04:47:33.189692 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [parameters: {'user_id': 'f5b0be3c0ebb494a96f094cb05458312', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 26 04:47:33.189692 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80206) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 26 04:47:33.201665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-f86d8b84-3f96-47b1-9dd8-8d14b7dd7b72 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 26 04:47:33.201665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:33.201665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 04:47:33.201665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 04:47:33.201665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 04:47:33.201665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 04:47:33.201665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 04:47:33.201665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 04:47:33.201665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 04:47:33.201665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application conn.query(q) Jul 26 04:47:33.201665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 04:47:33.201665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 04:47:33.201665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 04:47:33.201665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result.read() Jul 26 04:47:33.201665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 04:47:33.202720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 04:47:33.202720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 04:47:33.202720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 04:47:33.202720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 04:47:33.202720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 04:47:33.202720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 04:47:33.202720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 04:47:33.202720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 26 04:47:33.202720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:33.202720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 26 04:47:33.202720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:33.202720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:33.202720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 26 04:47:33.202720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 26 04:47:33.202720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 26 04:47:33.202720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 26 04:47:33.202720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 04:47:33.202720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application next(self.gen) Jul 26 04:47:33.203774 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 26 04:47:33.203774 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application yield resource Jul 26 04:47:33.203774 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 04:47:33.203774 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application next(self.gen) Jul 26 04:47:33.203774 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 26 04:47:33.203774 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.session.rollback() Jul 26 04:47:33.203774 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 26 04:47:33.203774 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.force_reraise() Jul 26 04:47:33.203774 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 26 04:47:33.203774 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise self.value Jul 26 04:47:33.203774 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 26 04:47:33.203774 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 26 04:47:33.203774 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 26 04:47:33.203774 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application session.commit() Jul 26 04:47:33.203774 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 26 04:47:33.203774 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 26 04:47:33.203774 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 26 04:47:33.204660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._prepare_impl() Jul 26 04:47:33.204660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 26 04:47:33.204660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.session.flush() Jul 26 04:47:33.204660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 26 04:47:33.204660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._flush(objects) Jul 26 04:47:33.204660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 26 04:47:33.204660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 26 04:47:33.204660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 26 04:47:33.204660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application compat.raise_( Jul 26 04:47:33.204660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 04:47:33.204660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception Jul 26 04:47:33.204660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 26 04:47:33.204660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application flush_context.execute() Jul 26 04:47:33.204660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 26 04:47:33.204660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rec.execute(self) Jul 26 04:47:33.204660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 26 04:47:33.204660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 26 04:47:33.211646 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 26 04:47:33.211646 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 26 04:47:33.211646 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 26 04:47:33.211646 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 26 04:47:33.211646 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 26 04:47:33.211646 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 26 04:47:33.211646 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 26 04:47:33.211646 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 26 04:47:33.211646 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 26 04:47:33.211646 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 26 04:47:33.211646 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 26 04:47:33.211646 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 26 04:47:33.211646 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 26 04:47:33.211646 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 26 04:47:33.211646 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 04:47:33.211646 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception Jul 26 04:47:33.213746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 04:47:33.213746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 04:47:33.213746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 04:47:33.213746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 04:47:33.213746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 04:47:33.213746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 04:47:33.213746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 04:47:33.213746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application conn.query(q) Jul 26 04:47:33.213746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 04:47:33.213746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 04:47:33.213746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 04:47:33.213746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result.read() Jul 26 04:47:33.213746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 04:47:33.213746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 04:47:33.213746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 04:47:33.213746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 04:47:33.216292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 04:47:33.216292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 04:47:33.216292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 04:47:33.216292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 04:47:33.216292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 26 04:47:33.216292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 26 04:47:33.216292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [parameters: {'user_id': 'f5b0be3c0ebb494a96f094cb05458312', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 26 04:47:33.216292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 26 04:47:33.216292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:33.216292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:47:33.216292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:33.216292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:33.216292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:33.216292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:33.216292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:33.218152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:33.218152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:33.218152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:33.218152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:33.218152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:33.218152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:33.218152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:33.218152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 26 04:47:33.218152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 26 04:47:33.218152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:33.218152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:33.218152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:47:33.218152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:33.218152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:47:33.218152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:33.218152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 26 04:47:33.218152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 26 04:47:33.221090 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 26 04:47:33.221090 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 26 04:47:33.221090 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 26 04:47:33.221090 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 26 04:47:33.221090 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 26 04:47:33.221090 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:33.221090 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 471/962] 10.222.0.17 () {60 vars in 1131 bytes} [Tue Jul 26 04:47:32 2022] POST /identity/v3/users => generated 160 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:47:33.221090 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d8b84-3f96-47b1-9dd8-8d14b7dd7b72 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:33.227982 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1b0c2-98ab-455d-9bb8-e179ec5e416d None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:33.272867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19ceaf-b9da-41e9-bd65-8ac433920bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:33.274015 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19ceaf-b9da-41e9-bd65-8ac433920bbb None admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:33.274875 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19ceaf-b9da-41e9-bd65-8ac433920bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:33.275458 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19ceaf-b9da-41e9-bd65-8ac433920bbb None admin] PATH_INFO: `/v3/projects/d583e0be8ef74766bffd6c3c07af18f1/users/3a8f3eb811ae40ad9cb64261f95a992e/roles/29516d985a494f4b88d90e1e4f525c24` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:33.278420 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf19ceaf-b9da-41e9-bd65-8ac433920bbb None admin] RBAC: Authorizing `identity:create_grant(project_id=d583e0be8ef74766bffd6c3c07af18f1, user_id=3a8f3eb811ae40ad9cb64261f95a992e, role_id=29516d985a494f4b88d90e1e4f525c24)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:33.280123 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf19ceaf-b9da-41e9-bd65-8ac433920bbb None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:33.299404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 492/963] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jul 26 04:47:33 2022] PUT /identity/v3/projects/d583e0be8ef74766bffd6c3c07af18f1/users/3a8f3eb811ae40ad9cb64261f95a992e/roles/29516d985a494f4b88d90e1e4f525c24 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:47:33.346461 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d8b84-3f96-47b1-9dd8-8d14b7dd7b72 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:33.399150 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e93f8-68e4-4427-bb4a-90a1427bb6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:33.400849 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e93f8-68e4-4427-bb4a-90a1427bb6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:33.401363 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e93f8-68e4-4427-bb4a-90a1427bb6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:33.401869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e93f8-68e4-4427-bb4a-90a1427bb6c8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:33.411899 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872e93f8-68e4-4427-bb4a-90a1427bb6c8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:33.414109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872e93f8-68e4-4427-bb4a-90a1427bb6c8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:33.428025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-872e93f8-68e4-4427-bb4a-90a1427bb6c8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 04:47:33.428025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:33.428025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:33.428025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:33.428025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:33.428025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:33.428025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:33.428025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:33.428025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:33.428025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:33.428025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:33.428025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:33.428025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:47:33.428025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:47:33.428025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:47:33.428025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:47:33.429110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:33.429110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:33.429110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:47:33.429110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:33.429110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:47:33.429110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:33.429110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:33.429110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:33.429110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:33.429110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:33.429110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:33.429110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:33.429110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:33.429110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:33.429110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:33.429110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:33.429110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:33.429110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:33.430127 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:33.430127 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:33.430127 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:47:33.430127 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:47:33.430127 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:47:33.430127 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:47:33.430127 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:47:33.430127 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:47:33.430127 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 04:47:33.430127 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:33.432696 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 472/964] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:47:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:47:33.443752 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19ceaf-b9da-41e9-bd65-8ac433920bbb None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:33.518035 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:33.554947 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19ceaf-b9da-41e9-bd65-8ac433920bbb None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:33.590205 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6d51d-1e68-4625-8461-34ea9bf3b8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:33.590871 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6d51d-1e68-4625-8461-34ea9bf3b8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:33.591347 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6d51d-1e68-4625-8461-34ea9bf3b8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:33.591827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6d51d-1e68-4625-8461-34ea9bf3b8b2 None admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:33.592622 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6d51d-1e68-4625-8461-34ea9bf3b8b2 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:33.592935 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6d51d-1e68-4625-8461-34ea9bf3b8b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:33.594407 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6d51d-1e68-4625-8461-34ea9bf3b8b2 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:33.632559 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 493/965] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jul 26 04:47:33 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:34.029071 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511de366-7a90-46b7-91d2-7f157370b551 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:34.029279 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511de366-7a90-46b7-91d2-7f157370b551 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:34.029669 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511de366-7a90-46b7-91d2-7f157370b551 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:34.033979 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 473/966] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:34.039307 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595753b-2c27-4ede-abdf-3e44dfe29080 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:34.039642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595753b-2c27-4ede-abdf-3e44dfe29080 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:34.039945 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595753b-2c27-4ede-abdf-3e44dfe29080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:34.059660 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-8595753b-2c27-4ede-abdf-3e44dfe29080 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:34.060879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-8595753b-2c27-4ede-abdf-3e44dfe29080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:34.137414 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 494/967] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:34.159266 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511de366-7a90-46b7-91d2-7f157370b551 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:34.184222 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:34.273975 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511de366-7a90-46b7-91d2-7f157370b551 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:34.314724 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9a0d3-0cd4-4fdd-9471-f5537fab2310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:34.315274 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9a0d3-0cd4-4fdd-9471-f5537fab2310 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:34.315527 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9a0d3-0cd4-4fdd-9471-f5537fab2310 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:34.315875 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9a0d3-0cd4-4fdd-9471-f5537fab2310 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:34.316588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe9a0d3-0cd4-4fdd-9471-f5537fab2310 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:34.317984 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe9a0d3-0cd4-4fdd-9471-f5537fab2310 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:34.343226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 474/968] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jul 26 04:47:34 2022] POST /identity/v3/services => generated 242 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:47:35.004415 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3ee08-51eb-4e5b-bef2-ac8b494e309a None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:35.005224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3ee08-51eb-4e5b-bef2-ac8b494e309a None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:35.005224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3ee08-51eb-4e5b-bef2-ac8b494e309a None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:35.006578 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 495/969] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:47:35.012086 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924bf67-c157-491d-a179-830f0dec276f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:35.012603 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924bf67-c157-491d-a179-830f0dec276f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:35.012603 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924bf67-c157-491d-a179-830f0dec276f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:35.031982 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9924bf67-c157-491d-a179-830f0dec276f None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:35.033538 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9924bf67-c157-491d-a179-830f0dec276f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:35.116476 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 475/970] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:35.139468 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3ee08-51eb-4e5b-bef2-ac8b494e309a None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:35.205871 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:35.286279 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3ee08-51eb-4e5b-bef2-ac8b494e309a None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:35.332675 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d03cfb-c020-41bc-8f0e-f966645c51ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:35.336055 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d03cfb-c020-41bc-8f0e-f966645c51ca None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:35.336055 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d03cfb-c020-41bc-8f0e-f966645c51ca None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:35.336055 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d03cfb-c020-41bc-8f0e-f966645c51ca None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:35.336055 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d03cfb-c020-41bc-8f0e-f966645c51ca None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:35.336055 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d03cfb-c020-41bc-8f0e-f966645c51ca None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:35.341075 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee71697-ebca-476c-8016-3869e114b161 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:35.341357 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee71697-ebca-476c-8016-3869e114b161 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:35.341617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee71697-ebca-476c-8016-3869e114b161 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:35.343131 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 476/971] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:47:35.348006 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9ad50-3eac-4177-b55c-c1a016abe44f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:35.348342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9ad50-3eac-4177-b55c-c1a016abe44f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:35.348577 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9ad50-3eac-4177-b55c-c1a016abe44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:35.348867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-77d03cfb-c020-41bc-8f0e-f966645c51ca None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 04:47:35.348867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:35.348867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 04:47:35.348867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 04:47:35.348867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 04:47:35.348867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 04:47:35.348867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 04:47:35.348867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:35.348867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 04:47:35.348867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:35.348867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:47:35.348867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:35.348867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:35.348867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:35.348867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:35.348867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:35.348867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:35.349959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:35.349959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:35.349959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:35.349959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:35.349959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:35.349959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:35.349959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 04:47:35.349959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 04:47:35.349959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 04:47:35.349959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 04:47:35.349959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:35.349959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:35.349959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:35.349959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:35.349959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:35.349959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:35.350850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:35.350850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:35.350850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:35.350850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:35.350850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:35.350850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:35.350850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:35.350850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:35.350850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:35.350850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:35.350850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 04:47:35.350850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:35.350850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 04:47:35.350850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:35.354996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 496/972] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Jul 26 04:47:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:35.356263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d03cfb-c020-41bc-8f0e-f966645c51ca None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:35.361929 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:35.369925 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1fd9ad50-3eac-4177-b55c-c1a016abe44f None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:35.371211 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1fd9ad50-3eac-4177-b55c-c1a016abe44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:35.445140 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 477/973] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 04:47:35.468814 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9ad50-3eac-4177-b55c-c1a016abe44f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:35.480037 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d03cfb-c020-41bc-8f0e-f966645c51ca None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:35.498351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:35.517257 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b32e0-34ba-4356-a5db-4d45bd297765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:35.517887 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b32e0-34ba-4356-a5db-4d45bd297765 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:35.518174 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b32e0-34ba-4356-a5db-4d45bd297765 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:35.518456 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b32e0-34ba-4356-a5db-4d45bd297765 None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:35.519146 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b32e0-34ba-4356-a5db-4d45bd297765 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:35.519453 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b32e0-34ba-4356-a5db-4d45bd297765 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:35.520873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b32e0-34ba-4356-a5db-4d45bd297765 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:35.526389 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 497/974] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Jul 26 04:47:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:35.531845 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b32e0-34ba-4356-a5db-4d45bd297765 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:35.584432 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9ad50-3eac-4177-b55c-c1a016abe44f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:35.622176 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543eb9de-70c0-42f4-93a1-121084dd7e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:35.622959 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543eb9de-70c0-42f4-93a1-121084dd7e18 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:35.623310 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543eb9de-70c0-42f4-93a1-121084dd7e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:35.623686 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543eb9de-70c0-42f4-93a1-121084dd7e18 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:35.626919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:35.638503 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543eb9de-70c0-42f4-93a1-121084dd7e18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:47:35.645712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-543eb9de-70c0-42f4-93a1-121084dd7e18 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 04:47:35.645712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:35.645712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:35.645712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:35.645712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:35.645712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:35.645712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:35.645712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:35.645712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:35.645712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:35.645712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:35.645712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:35.645712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:47:35.645712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:47:35.645712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:47:35.645712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:47:35.646713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:47:35.646713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:47:35.646713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:47:35.646713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:47:35.646713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:35.646713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:35.646713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:35.646713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:35.646713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:35.646713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:35.646713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:35.646713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:35.646713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:35.646713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:35.646713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:35.646713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:35.653019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:35.653019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:35.653019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:35.653019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:35.653019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:47:35.653019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:47:35.653019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:47:35.653019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:47:35.653019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:47:35.653019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:47:35.653019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 04:47:35.653019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:35.653019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 478/975] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Jul 26 04:47:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:47:35.653796 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b32e0-34ba-4356-a5db-4d45bd297765 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:35.655860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543eb9de-70c0-42f4-93a1-121084dd7e18 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:35.683773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc671ea-f79a-4bc5-8252-dd6adb465eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:35.684422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc671ea-f79a-4bc5-8252-dd6adb465eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:35.684683 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc671ea-f79a-4bc5-8252-dd6adb465eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:35.684952 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc671ea-f79a-4bc5-8252-dd6adb465eb1 None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:35.685634 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc671ea-f79a-4bc5-8252-dd6adb465eb1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:35.685937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc671ea-f79a-4bc5-8252-dd6adb465eb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:35.687312 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc671ea-f79a-4bc5-8252-dd6adb465eb1 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:35.698001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 498/976] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Jul 26 04:47:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:35.699590 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc671ea-f79a-4bc5-8252-dd6adb465eb1 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:35.748476 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:35.769851 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543eb9de-70c0-42f4-93a1-121084dd7e18 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:35.808580 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aece57-f558-4f22-931b-4416dd99de01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:35.808839 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc671ea-f79a-4bc5-8252-dd6adb465eb1 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:35.809216 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aece57-f558-4f22-931b-4416dd99de01 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:35.809499 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aece57-f558-4f22-931b-4416dd99de01 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:35.809772 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aece57-f558-4f22-931b-4416dd99de01 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:35.810490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aece57-f558-4f22-931b-4416dd99de01 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:35.810799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aece57-f558-4f22-931b-4416dd99de01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:35.812216 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aece57-f558-4f22-931b-4416dd99de01 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:35.821459 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 479/977] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Jul 26 04:47:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:35.827118 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aece57-f558-4f22-931b-4416dd99de01 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:35.846907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3e500-5a5d-494e-8834-22a286e4af55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:35.847895 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3e500-5a5d-494e-8834-22a286e4af55 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:35.848282 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3e500-5a5d-494e-8834-22a286e4af55 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:35.848654 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3e500-5a5d-494e-8834-22a286e4af55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:35.849540 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa3e500-5a5d-494e-8834-22a286e4af55 None admin] RBAC: Adding query filter params (interface=admin, service_id=7f16dcb1cca742ddaefab6bf1923efeb, region_id=RegionOne) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:35.849956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa3e500-5a5d-494e-8834-22a286e4af55 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:35.851914 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa3e500-5a5d-494e-8834-22a286e4af55 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:35.858375 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 499/978] 10.222.0.17 () {56 vars in 1249 bytes} [Tue Jul 26 04:47:35 2022] GET /identity/v3/endpoints?service_id=7f16dcb1cca742ddaefab6bf1923efeb&interface=admin®ion_id=RegionOne => generated 193 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:35.863840 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3e500-5a5d-494e-8834-22a286e4af55 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:35.889183 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:35.934287 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aece57-f558-4f22-931b-4416dd99de01 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:35.968603 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e196e35-5d29-4589-bb99-61bf828fc9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:35.969433 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e196e35-5d29-4589-bb99-61bf828fc9de None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:35.969783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e196e35-5d29-4589-bb99-61bf828fc9de None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:35.970178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e196e35-5d29-4589-bb99-61bf828fc9de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:35.972419 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3e500-5a5d-494e-8834-22a286e4af55 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:35.981838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e196e35-5d29-4589-bb99-61bf828fc9de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:35.983427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e196e35-5d29-4589-bb99-61bf828fc9de None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:35.984191 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:35.999816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-1e196e35-5d29-4589-bb99-61bf828fc9de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:35.999816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:35.999816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:35.999816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:35.999816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:35.999816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:35.999816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:35.999816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:35.999816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:35.999816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:35.999816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:35.999816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:35.999816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:35.999816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:35.999816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:36.001074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:36.001074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:36.001074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:36.001074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:36.001074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:36.001074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:36.001074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:36.001074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:36.001074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:36.001074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:36.001074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:36.001074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:36.001074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:36.001074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:36.001074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:36.001074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:36.001074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:36.002154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:36.002154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:36.002154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:36.002154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:36.002729 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 480/979] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:47:36.008689 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e196e35-5d29-4589-bb99-61bf828fc9de None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:36.019874 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c163ce-16db-4320-bbaf-a0239a54227f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:36.020539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c163ce-16db-4320-bbaf-a0239a54227f None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:36.020793 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c163ce-16db-4320-bbaf-a0239a54227f None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:36.021078 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c163ce-16db-4320-bbaf-a0239a54227f None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:36.021723 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c163ce-16db-4320-bbaf-a0239a54227f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:36.023111 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c163ce-16db-4320-bbaf-a0239a54227f None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:36.030095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 500/980] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Jul 26 04:47:35 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 04:47:36.150582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e196e35-5d29-4589-bb99-61bf828fc9de None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:36.162909 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:36.191171 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d369d-ec4c-4d59-9cbd-f592a3d52102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:36.191930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d369d-ec4c-4d59-9cbd-f592a3d52102 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:36.192312 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d369d-ec4c-4d59-9cbd-f592a3d52102 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:36.193695 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d369d-ec4c-4d59-9cbd-f592a3d52102 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:36.193695 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d369d-ec4c-4d59-9cbd-f592a3d52102 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:36.193695 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d369d-ec4c-4d59-9cbd-f592a3d52102 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:36.195141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d369d-ec4c-4d59-9cbd-f592a3d52102 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:36.208659 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 481/981] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:36.214722 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c163ce-16db-4320-bbaf-a0239a54227f None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:36.326636 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c163ce-16db-4320-bbaf-a0239a54227f None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:36.348310 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8b517-f122-408f-9e40-74023d0773e6 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:36.348654 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8b517-f122-408f-9e40-74023d0773e6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:36.348882 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8b517-f122-408f-9e40-74023d0773e6 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:36.350482 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 482/982] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:36.356021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8980a-d03a-4502-a29e-eba682453ddd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:36.356309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8980a-d03a-4502-a29e-eba682453ddd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:36.356596 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8980a-d03a-4502-a29e-eba682453ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:36.366494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c099510-0c38-461a-98f3-ed030e3f7754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:36.367145 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c099510-0c38-461a-98f3-ed030e3f7754 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:36.367458 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c099510-0c38-461a-98f3-ed030e3f7754 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:36.367731 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c099510-0c38-461a-98f3-ed030e3f7754 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:36.375450 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e3f8980a-d03a-4502-a29e-eba682453ddd None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:36.376140 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c099510-0c38-461a-98f3-ed030e3f7754 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:36.376701 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e3f8980a-d03a-4502-a29e-eba682453ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:36.377675 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c099510-0c38-461a-98f3-ed030e3f7754 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:36.392785 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-6c099510-0c38-461a-98f3-ed030e3f7754 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 04:47:36.392785 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:36.392785 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:36.392785 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:36.392785 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:36.392785 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:36.392785 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:36.392785 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:36.392785 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:36.392785 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:36.392785 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:36.392785 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:36.392785 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:47:36.392785 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:47:36.392785 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:47:36.392785 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:47:36.393733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:36.393733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:36.393733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:47:36.393733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:36.393733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:47:36.393733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:36.393733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:36.393733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:36.393733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:36.393733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:36.393733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:36.393733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:36.393733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:36.393733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:36.393733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:36.393733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:36.393733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:36.393733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:36.394664 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:36.394664 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:36.394664 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:47:36.394664 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:47:36.394664 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:47:36.394664 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:47:36.394664 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:47:36.394664 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:47:36.394664 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 04:47:36.394664 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:36.394664 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 501/983] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Jul 26 04:47:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:36.399805 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c099510-0c38-461a-98f3-ed030e3f7754 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:36.436460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:36.461243 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 483/984] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:36.483764 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8980a-d03a-4502-a29e-eba682453ddd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:36.509245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c099510-0c38-461a-98f3-ed030e3f7754 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:36.547010 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf0f0a-2505-42ab-a9da-ec0b1907fe39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:36.547653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf0f0a-2505-42ab-a9da-ec0b1907fe39 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:36.547935 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf0f0a-2505-42ab-a9da-ec0b1907fe39 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:36.548211 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf0f0a-2505-42ab-a9da-ec0b1907fe39 None admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:36.549020 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebf0f0a-2505-42ab-a9da-ec0b1907fe39 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:36.549333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebf0f0a-2505-42ab-a9da-ec0b1907fe39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:36.550880 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebf0f0a-2505-42ab-a9da-ec0b1907fe39 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:36.569778 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 502/985] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Jul 26 04:47:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:36.575610 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf0f0a-2505-42ab-a9da-ec0b1907fe39 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:36.597537 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:36.601900 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8980a-d03a-4502-a29e-eba682453ddd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:36.638388 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6772f18-6e27-426b-a538-37f242258da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:36.639037 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6772f18-6e27-426b-a538-37f242258da3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:36.639315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6772f18-6e27-426b-a538-37f242258da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:36.639614 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6772f18-6e27-426b-a538-37f242258da3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:36.640242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6772f18-6e27-426b-a538-37f242258da3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:36.641659 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6772f18-6e27-426b-a538-37f242258da3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:36.650838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a6772f18-6e27-426b-a538-37f242258da3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 04:47:36.650838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:36.650838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 04:47:36.650838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 04:47:36.650838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 04:47:36.650838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 04:47:36.650838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 04:47:36.650838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:36.650838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 04:47:36.650838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:36.650838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:47:36.650838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:36.650838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:36.650838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:36.650838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:36.650838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:36.650838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:36.652146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:36.652146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:36.652146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:36.652146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:36.652146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:36.652146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:36.652146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 04:47:36.652146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 04:47:36.652146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 04:47:36.652146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 04:47:36.652146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:36.652146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:36.652146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:36.652146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:36.652146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:36.652146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:36.652991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:36.652991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:36.652991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:36.652991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:36.652991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:36.652991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:36.652991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:36.652991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:36.652991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:36.652991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:36.652991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 04:47:36.652991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:36.652991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 04:47:36.652991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:36.652991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 484/986] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:47:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:47:36.658012 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6772f18-6e27-426b-a538-37f242258da3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:36.678144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:36.688186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf0f0a-2505-42ab-a9da-ec0b1907fe39 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:36.725837 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f1e7d-eb56-411a-9551-e9bc01af84c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:36.726458 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f1e7d-eb56-411a-9551-e9bc01af84c7 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:36.726720 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f1e7d-eb56-411a-9551-e9bc01af84c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:36.726988 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f1e7d-eb56-411a-9551-e9bc01af84c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:36.764599 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6772f18-6e27-426b-a538-37f242258da3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:36.799502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530f403-96b0-422c-963b-7449d2b7bf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:36.800177 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530f403-96b0-422c-963b-7449d2b7bf71 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:36.800463 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530f403-96b0-422c-963b-7449d2b7bf71 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:36.800739 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530f403-96b0-422c-963b-7449d2b7bf71 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:36.801430 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9530f403-96b0-422c-963b-7449d2b7bf71 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:36.801740 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9530f403-96b0-422c-963b-7449d2b7bf71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:36.802966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f1e7d-eb56-411a-9551-e9bc01af84c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:36.803125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9530f403-96b0-422c-963b-7449d2b7bf71 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:36.805359 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f1e7d-eb56-411a-9551-e9bc01af84c7 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:36.808490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 485/987] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Jul 26 04:47:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:36.814164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530f403-96b0-422c-963b-7449d2b7bf71 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:36.844202 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:36.904105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:36.919777 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 503/988] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:47:36 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 04:47:36.920871 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f1e7d-eb56-411a-9551-e9bc01af84c7 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:36.928847 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530f403-96b0-422c-963b-7449d2b7bf71 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:36.966100 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a72ff04-d457-4d0d-9569-9c0af5502a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:36.966668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72ff04-d457-4d0d-9569-9c0af5502a72 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:36.966946 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72ff04-d457-4d0d-9569-9c0af5502a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:36.967218 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72ff04-d457-4d0d-9569-9c0af5502a72 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:36.967931 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a72ff04-d457-4d0d-9569-9c0af5502a72 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:36.968231 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a72ff04-d457-4d0d-9569-9c0af5502a72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:36.969596 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a72ff04-d457-4d0d-9569-9c0af5502a72 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:36.975440 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 486/989] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Jul 26 04:47:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:36.981302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a72ff04-d457-4d0d-9569-9c0af5502a72 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:37.030626 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f1e7d-eb56-411a-9551-e9bc01af84c7 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:37.069314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02bdd6-c560-4e53-a6e9-845e1f2f099d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:37.070250 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02bdd6-c560-4e53-a6e9-845e1f2f099d None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:37.070529 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02bdd6-c560-4e53-a6e9-845e1f2f099d None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:37.070806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02bdd6-c560-4e53-a6e9-845e1f2f099d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:37.076159 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:37.083471 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02bdd6-c560-4e53-a6e9-845e1f2f099d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:37.085021 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02bdd6-c560-4e53-a6e9-845e1f2f099d None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:37.087165 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a72ff04-d457-4d0d-9569-9c0af5502a72 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:37.101326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-ce02bdd6-c560-4e53-a6e9-845e1f2f099d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:37.101326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:37.101326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:37.101326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:37.101326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:37.101326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:37.101326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:37.101326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:37.101326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:37.101326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:37.101326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:37.101326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:37.101326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:37.101326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:37.101326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:37.102385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:37.102385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:37.102385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:37.102385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:37.102385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:37.102385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:37.102385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:37.102385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:37.102385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:37.102385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:37.102385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:37.102385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:37.102385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:37.102385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:37.102385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:37.102385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:37.102385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:37.103366 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:37.103366 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:37.103366 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:37.103366 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:37.103366 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 504/990] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:37.108256 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02bdd6-c560-4e53-a6e9-845e1f2f099d None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:37.127427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e303983-ff92-4aa3-bede-913e7068ea00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:37.128275 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e303983-ff92-4aa3-bede-913e7068ea00 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:37.128557 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e303983-ff92-4aa3-bede-913e7068ea00 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:37.128878 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e303983-ff92-4aa3-bede-913e7068ea00 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:37.129689 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e303983-ff92-4aa3-bede-913e7068ea00 None admin] RBAC: Adding query filter params (interface=public, service_id=bb96c80799454a73b8fa1e9e121ca638, region_id=RegionOne) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:37.129997 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e303983-ff92-4aa3-bede-913e7068ea00 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:37.131432 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e303983-ff92-4aa3-bede-913e7068ea00 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:37.137336 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 487/991] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Jul 26 04:47:36 2022] GET /identity/v3/endpoints?service_id=bb96c80799454a73b8fa1e9e121ca638&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:37.142587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e303983-ff92-4aa3-bede-913e7068ea00 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:37.166347 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:37.228898 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02bdd6-c560-4e53-a6e9-845e1f2f099d None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:37.257951 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e303983-ff92-4aa3-bede-913e7068ea00 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:37.263951 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c15b05-a77d-4dc0-ab70-2be795781cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:37.264584 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c15b05-a77d-4dc0-ab70-2be795781cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:37.264866 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c15b05-a77d-4dc0-ab70-2be795781cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:37.265137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c15b05-a77d-4dc0-ab70-2be795781cd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:37.265832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c15b05-a77d-4dc0-ab70-2be795781cd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:37.266134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c15b05-a77d-4dc0-ab70-2be795781cd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:37.267762 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c15b05-a77d-4dc0-ab70-2be795781cd8 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:37.285357 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 505/992] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:37.286753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c15b05-a77d-4dc0-ab70-2be795781cd8 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:37.293310 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97bba8b-4fca-44a9-838f-05bb47e8fc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:37.293942 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97bba8b-4fca-44a9-838f-05bb47e8fc87 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:37.294252 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97bba8b-4fca-44a9-838f-05bb47e8fc87 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:37.294488 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97bba8b-4fca-44a9-838f-05bb47e8fc87 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:37.295139 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97bba8b-4fca-44a9-838f-05bb47e8fc87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:37.296606 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97bba8b-4fca-44a9-838f-05bb47e8fc87 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:37.303623 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 488/993] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Jul 26 04:47:37 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 04:47:37.413300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c15b05-a77d-4dc0-ab70-2be795781cd8 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:37.416914 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:37.457157 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43ca28-ff88-40e6-8c77-765231006370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:37.457413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43ca28-ff88-40e6-8c77-765231006370 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:37.457643 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43ca28-ff88-40e6-8c77-765231006370 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:37.457851 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43ca28-ff88-40e6-8c77-765231006370 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:37.468056 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43ca28-ff88-40e6-8c77-765231006370 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:37.469547 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43ca28-ff88-40e6-8c77-765231006370 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:37.482338 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-4b43ca28-ff88-40e6-8c77-765231006370 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:37.482338 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:37.482338 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:37.482338 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:37.482338 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:37.482338 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:37.482338 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:37.482338 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:37.482338 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:37.482338 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:37.482338 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:37.482338 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:37.482338 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:47:37.482338 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:47:37.482338 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:47:37.483265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:47:37.483265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:37.483265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:37.483265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:37.483265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:37.483265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:37.483265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:37.483265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:37.483265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:37.483265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:37.483265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:37.483265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:37.483265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:37.483265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:37.483265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:37.483265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:37.483265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:37.484227 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:47:37.484227 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:47:37.484227 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:47:37.484227 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:47:37.484227 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:47:37.484227 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:47:37.484227 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:37.484227 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:37.484227 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 506/994] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:47:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:47:37.491346 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97bba8b-4fca-44a9-838f-05bb47e8fc87 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:37.521425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:37.606990 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97bba8b-4fca-44a9-838f-05bb47e8fc87 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:37.642177 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbffeab-8025-479c-80be-9193ee136010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:37.642819 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbffeab-8025-479c-80be-9193ee136010 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:37.643103 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbffeab-8025-479c-80be-9193ee136010 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:37.643377 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbffeab-8025-479c-80be-9193ee136010 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:37.644434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbffeab-8025-479c-80be-9193ee136010 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:37.644543 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbffeab-8025-479c-80be-9193ee136010 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:37.645954 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbffeab-8025-479c-80be-9193ee136010 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:37.659546 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 489/995] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:47:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:37.666031 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43ca28-ff88-40e6-8c77-765231006370 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:37.770901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43ca28-ff88-40e6-8c77-765231006370 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:37.808381 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc862c-a2b4-4ffe-957d-061bac5efca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:37.809020 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc862c-a2b4-4ffe-957d-061bac5efca1 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:37.809306 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc862c-a2b4-4ffe-957d-061bac5efca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:37.809580 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc862c-a2b4-4ffe-957d-061bac5efca1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:37.810380 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cc862c-a2b4-4ffe-957d-061bac5efca1 None admin] RBAC: Adding query filter params (role.id=29516d985a494f4b88d90e1e4f525c24, scope.project.id=d583e0be8ef74766bffd6c3c07af18f1, user.id=3a8f3eb811ae40ad9cb64261f95a992e) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:37.810692 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cc862c-a2b4-4ffe-957d-061bac5efca1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:37.812202 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cc862c-a2b4-4ffe-957d-061bac5efca1 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:37.823815 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 507/996] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:47:37 2022] GET /identity/v3/role_assignments?user.id=3a8f3eb811ae40ad9cb64261f95a992e&scope.project.id=d583e0be8ef74766bffd6c3c07af18f1&role.id=29516d985a494f4b88d90e1e4f525c24 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:38.097142 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566dee6-f349-426f-bf72-9e5618c670d9 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:38.097343 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566dee6-f349-426f-bf72-9e5618c670d9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:38.097671 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566dee6-f349-426f-bf72-9e5618c670d9 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:38.099175 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 490/997] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:38.104639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897eb606-9412-4fdb-b6f5-2d8dad0b9e0d None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:38.105124 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897eb606-9412-4fdb-b6f5-2d8dad0b9e0d None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:38.105201 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897eb606-9412-4fdb-b6f5-2d8dad0b9e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:38.124154 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-897eb606-9412-4fdb-b6f5-2d8dad0b9e0d None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:38.125358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-897eb606-9412-4fdb-b6f5-2d8dad0b9e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:38.149308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:38.197619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 508/998] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:38 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:38.220293 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566dee6-f349-426f-bf72-9e5618c670d9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:38.305054 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:38.327904 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566dee6-f349-426f-bf72-9e5618c670d9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:38.366987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fe31c-4981-40c2-91da-cf6abd252056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:38.367405 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fe31c-4981-40c2-91da-cf6abd252056 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:38.367745 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fe31c-4981-40c2-91da-cf6abd252056 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:38.368021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fe31c-4981-40c2-91da-cf6abd252056 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:38.368683 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fe31c-4981-40c2-91da-cf6abd252056 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:38.370064 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fe31c-4981-40c2-91da-cf6abd252056 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:38.379705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d39fe31c-4981-40c2-91da-cf6abd252056 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 04:47:38.379705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:38.379705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 04:47:38.379705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 04:47:38.379705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 04:47:38.379705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 04:47:38.379705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 04:47:38.379705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:38.379705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 04:47:38.379705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:38.379705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:47:38.379705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:38.379705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:38.379705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:38.379705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:38.379705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:38.379705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:38.380723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:38.380723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:38.380723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:38.380723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:38.380723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:38.380723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:38.380723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 04:47:38.380723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 04:47:38.380723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 04:47:38.380723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 04:47:38.380723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:38.380723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:38.380723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:38.380723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:38.380723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:38.380723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:38.381693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:38.381693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:38.381693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:38.381693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:38.381693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:38.381693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:38.381693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:38.381693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:38.381693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:38.381693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:38.381693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 04:47:38.381693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:38.381693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 04:47:38.381693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:38.384107 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 491/999] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Jul 26 04:47:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:38.389040 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897eb606-9412-4fdb-b6f5-2d8dad0b9e0d None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:38.495615 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897eb606-9412-4fdb-b6f5-2d8dad0b9e0d None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:38.533226 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c166d70-47f0-4ec7-a3b9-cdd754eb0489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:38.533856 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c166d70-47f0-4ec7-a3b9-cdd754eb0489 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:38.534204 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c166d70-47f0-4ec7-a3b9-cdd754eb0489 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:38.534539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c166d70-47f0-4ec7-a3b9-cdd754eb0489 None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:38.535114 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c166d70-47f0-4ec7-a3b9-cdd754eb0489 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:38.535430 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c166d70-47f0-4ec7-a3b9-cdd754eb0489 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:38.536895 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c166d70-47f0-4ec7-a3b9-cdd754eb0489 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:38.542340 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 509/1000] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Jul 26 04:47:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:38.548187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fe31c-4981-40c2-91da-cf6abd252056 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:38.654187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fe31c-4981-40c2-91da-cf6abd252056 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:38.689143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94462c1d-ae9d-40b5-a341-d9ea159173b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:38.689792 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94462c1d-ae9d-40b5-a341-d9ea159173b5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:38.690065 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94462c1d-ae9d-40b5-a341-d9ea159173b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:38.690341 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94462c1d-ae9d-40b5-a341-d9ea159173b5 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:38.691028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94462c1d-ae9d-40b5-a341-d9ea159173b5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:38.691334 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94462c1d-ae9d-40b5-a341-d9ea159173b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:38.692748 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94462c1d-ae9d-40b5-a341-d9ea159173b5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:38.697956 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 492/1001] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Jul 26 04:47:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:38.703840 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c166d70-47f0-4ec7-a3b9-cdd754eb0489 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:38.736439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:38.814030 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c166d70-47f0-4ec7-a3b9-cdd754eb0489 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:38.849301 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dfe2b2-75fe-4b7c-9e05-278c6dd516d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:38.849927 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dfe2b2-75fe-4b7c-9e05-278c6dd516d8 None admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:38.850210 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dfe2b2-75fe-4b7c-9e05-278c6dd516d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:38.850484 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dfe2b2-75fe-4b7c-9e05-278c6dd516d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:38.851221 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dfe2b2-75fe-4b7c-9e05-278c6dd516d8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:38.852677 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dfe2b2-75fe-4b7c-9e05-278c6dd516d8 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:38.875591 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 510/1002] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 04:47:38 2022] POST /identity/v3/endpoints => generated 325 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:47:39.302030 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8621ff-9a7b-443c-9a25-aaeecd9a8e4a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:39.302293 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8621ff-9a7b-443c-9a25-aaeecd9a8e4a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:39.302687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8621ff-9a7b-443c-9a25-aaeecd9a8e4a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:39.304299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 493/1003] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:47:39.309706 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd94c5-afb7-4946-8b26-69c0663172c0 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:39.310428 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd94c5-afb7-4946-8b26-69c0663172c0 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:39.310428 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd94c5-afb7-4946-8b26-69c0663172c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:39.331415 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-09bd94c5-afb7-4946-8b26-69c0663172c0 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:39.332731 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-09bd94c5-afb7-4946-8b26-69c0663172c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:39.418340 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 511/1004] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:39 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:39.441919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8621ff-9a7b-443c-9a25-aaeecd9a8e4a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:39.454147 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:39.548505 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8621ff-9a7b-443c-9a25-aaeecd9a8e4a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:39.592206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61a7e5-dfc4-4c7f-8c16-1f300b4d61c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:39.592784 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61a7e5-dfc4-4c7f-8c16-1f300b4d61c1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:39.593067 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61a7e5-dfc4-4c7f-8c16-1f300b4d61c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:39.593344 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61a7e5-dfc4-4c7f-8c16-1f300b4d61c1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:39.593985 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e61a7e5-dfc4-4c7f-8c16-1f300b4d61c1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:39.595364 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e61a7e5-dfc4-4c7f-8c16-1f300b4d61c1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:39.605014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-5e61a7e5-dfc4-4c7f-8c16-1f300b4d61c1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 04:47:39.605014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:39.605014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 04:47:39.605014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 04:47:39.605014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 04:47:39.605014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 04:47:39.605014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 04:47:39.605014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:39.605014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 04:47:39.605014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:39.605014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:47:39.605014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:39.605014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:39.605014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:39.605014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:39.605014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:39.605014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:39.614987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:39.614987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:39.614987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:39.614987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:39.614987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:39.614987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:39.614987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 04:47:39.614987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 04:47:39.614987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 04:47:39.614987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 04:47:39.614987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:39.614987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:39.614987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:39.614987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:39.614987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:39.614987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:39.616080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:39.616080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:39.616080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:39.616080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:39.616080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:39.616080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:39.616080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:39.616080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:39.616080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:39.616080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:39.616080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 04:47:39.616080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:39.616080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 04:47:39.616080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:39.616080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 494/1005] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:47:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:47:39.617110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bd94c5-afb7-4946-8b26-69c0663172c0 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:39.623718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:39.719020 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bd94c5-afb7-4946-8b26-69c0663172c0 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:39.753030 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baee7b8-c170-44cb-ac6a-a15eb2f82373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:39.753730 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baee7b8-c170-44cb-ac6a-a15eb2f82373 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:39.754011 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baee7b8-c170-44cb-ac6a-a15eb2f82373 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:39.754281 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baee7b8-c170-44cb-ac6a-a15eb2f82373 None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:39.754962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baee7b8-c170-44cb-ac6a-a15eb2f82373 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:39.755265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baee7b8-c170-44cb-ac6a-a15eb2f82373 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:39.756679 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baee7b8-c170-44cb-ac6a-a15eb2f82373 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:39.766133 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 512/1006] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Jul 26 04:47:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:39.767814 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61a7e5-dfc4-4c7f-8c16-1f300b4d61c1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:39.845228 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:39.877428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61a7e5-dfc4-4c7f-8c16-1f300b4d61c1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:39.886684 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e8e2c-c37a-4323-aecb-3a5057c6e5a3 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:39.887594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e8e2c-c37a-4323-aecb-3a5057c6e5a3 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:39.888034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e8e2c-c37a-4323-aecb-3a5057c6e5a3 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:39.889850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 513/1007] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:39.894928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e2957-606f-44b8-9994-2b9bd6000d1d None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:39.895219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e2957-606f-44b8-9994-2b9bd6000d1d None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:39.895493 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e2957-606f-44b8-9994-2b9bd6000d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:39.913943 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-3c6e2957-606f-44b8-9994-2b9bd6000d1d None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:39.915286 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-3c6e2957-606f-44b8-9994-2b9bd6000d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:39.917474 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386c491-8a77-43aa-8c7c-cc58d78903b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:39.918904 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386c491-8a77-43aa-8c7c-cc58d78903b6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:39.919199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386c491-8a77-43aa-8c7c-cc58d78903b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:39.919476 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386c491-8a77-43aa-8c7c-cc58d78903b6 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:39.920349 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386c491-8a77-43aa-8c7c-cc58d78903b6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:39.920660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386c491-8a77-43aa-8c7c-cc58d78903b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:39.922042 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386c491-8a77-43aa-8c7c-cc58d78903b6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:39.927807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 495/1008] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Jul 26 04:47:39 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:39.933843 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386c491-8a77-43aa-8c7c-cc58d78903b6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:40.000216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 514/1009] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:39 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:40.022292 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e2957-606f-44b8-9994-2b9bd6000d1d None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:40.026232 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:40.053915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386c491-8a77-43aa-8c7c-cc58d78903b6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:40.097146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1efe035-9ffa-4c5d-935e-b93c7a1257ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:40.097693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1efe035-9ffa-4c5d-935e-b93c7a1257ec None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:40.098004 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1efe035-9ffa-4c5d-935e-b93c7a1257ec None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:40.098286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1efe035-9ffa-4c5d-935e-b93c7a1257ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:40.099009 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1efe035-9ffa-4c5d-935e-b93c7a1257ec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:40.100423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1efe035-9ffa-4c5d-935e-b93c7a1257ec None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:40.105991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:40.121872 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 496/1010] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 04:47:39 2022] POST /identity/v3/endpoints => generated 334 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 04:47:40.157322 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e2957-606f-44b8-9994-2b9bd6000d1d None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:40.204734 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14509ed4-d6e5-43e6-987b-93bcbc8f96d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:40.205403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14509ed4-d6e5-43e6-987b-93bcbc8f96d9 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:40.205682 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14509ed4-d6e5-43e6-987b-93bcbc8f96d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:40.205953 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14509ed4-d6e5-43e6-987b-93bcbc8f96d9 None admin] PATH_INFO: `/v3/services/image` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:40.206607 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14509ed4-d6e5-43e6-987b-93bcbc8f96d9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:40.208065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14509ed4-d6e5-43e6-987b-93bcbc8f96d9 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:40.218434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-14509ed4-d6e5-43e6-987b-93bcbc8f96d9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 04:47:40.218434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:40.218434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 04:47:40.218434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 04:47:40.218434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 04:47:40.218434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 04:47:40.218434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 04:47:40.218434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:40.218434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 04:47:40.218434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:40.218434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:47:40.218434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:40.218434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:40.218434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:40.218434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:40.218434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:40.218434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:40.219533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:40.219533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:40.219533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:40.219533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:40.219533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:40.219533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:40.219533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 04:47:40.219533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 04:47:40.219533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 04:47:40.219533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 04:47:40.219533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:40.219533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:40.219533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:40.219533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:40.219533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:40.219533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:40.220827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:40.220827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:40.220827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:40.220827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:40.220827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:40.220827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:40.220827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:40.220827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:40.220827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:40.220827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:40.220827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 04:47:40.220827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:40.220827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 04:47:40.220827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:40.222039 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 515/1011] 10.222.0.17 () {56 vars in 1105 bytes} [Tue Jul 26 04:47:40 2022] GET /identity/v3/services/image => generated 86 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:47:40.226090 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1efe035-9ffa-4c5d-935e-b93c7a1257ec None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:40.341770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1efe035-9ffa-4c5d-935e-b93c7a1257ec None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:40.381066 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1304b88-1b1c-4ecd-933c-97359d2a4de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:40.381713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1304b88-1b1c-4ecd-933c-97359d2a4de9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:40.382002 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1304b88-1b1c-4ecd-933c-97359d2a4de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:40.382279 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1304b88-1b1c-4ecd-933c-97359d2a4de9 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:40.382983 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1304b88-1b1c-4ecd-933c-97359d2a4de9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:40.383301 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1304b88-1b1c-4ecd-933c-97359d2a4de9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:40.386901 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1304b88-1b1c-4ecd-933c-97359d2a4de9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:40.390464 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 497/1012] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:40.396602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14509ed4-d6e5-43e6-987b-93bcbc8f96d9 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:40.463282 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:40.505905 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14509ed4-d6e5-43e6-987b-93bcbc8f96d9 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:40.541355 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd67d418-64f6-439c-95da-8acd5a3da796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:40.542037 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67d418-64f6-439c-95da-8acd5a3da796 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:40.542321 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67d418-64f6-439c-95da-8acd5a3da796 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:40.542594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67d418-64f6-439c-95da-8acd5a3da796 None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:40.543282 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd67d418-64f6-439c-95da-8acd5a3da796 None admin] RBAC: Adding query filter params (type=image) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:40.543601 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd67d418-64f6-439c-95da-8acd5a3da796 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:40.544998 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd67d418-64f6-439c-95da-8acd5a3da796 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:40.550229 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 516/1013] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:40 2022] GET /identity/v3/services?type=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:40.893942 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de77801-6aec-4294-a70d-50e77503db14 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:40.894474 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de77801-6aec-4294-a70d-50e77503db14 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:40.894474 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de77801-6aec-4294-a70d-50e77503db14 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:40.896099 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 498/1014] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:40.901652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95187b8d-8568-49f7-925c-b823557499bd None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:40.901943 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95187b8d-8568-49f7-925c-b823557499bd None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:40.902222 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95187b8d-8568-49f7-925c-b823557499bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:40.921510 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-95187b8d-8568-49f7-925c-b823557499bd None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:40.922731 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-95187b8d-8568-49f7-925c-b823557499bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:41.004509 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 517/1015] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:40 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:41.036741 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de77801-6aec-4294-a70d-50e77503db14 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:41.100342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:41.148934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de77801-6aec-4294-a70d-50e77503db14 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:41.184147 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64ddff-54bf-4b92-b5fc-229d4098f684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:41.184812 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64ddff-54bf-4b92-b5fc-229d4098f684 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:41.185096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64ddff-54bf-4b92-b5fc-229d4098f684 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:41.185378 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64ddff-54bf-4b92-b5fc-229d4098f684 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:41.186571 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c64ddff-54bf-4b92-b5fc-229d4098f684 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:41.188050 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c64ddff-54bf-4b92-b5fc-229d4098f684 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:41.189865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 499/1016] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:41.195947 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95187b8d-8568-49f7-925c-b823557499bd None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:41.257740 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:41.311257 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95187b8d-8568-49f7-925c-b823557499bd None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:41.353826 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b94eb2e-047b-4c83-b428-5ab42840b098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:41.354549 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94eb2e-047b-4c83-b428-5ab42840b098 None admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:41.354828 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94eb2e-047b-4c83-b428-5ab42840b098 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:41.355156 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94eb2e-047b-4c83-b428-5ab42840b098 None admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:41.355906 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b94eb2e-047b-4c83-b428-5ab42840b098 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:41.357449 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b94eb2e-047b-4c83-b428-5ab42840b098 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:41.399084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 518/1017] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jul 26 04:47:41 2022] POST /identity/v3/projects => generated 311 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:47:42.271766 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba72162-51d7-45dd-bf87-a18b04632fb2 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:42.272374 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba72162-51d7-45dd-bf87-a18b04632fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:42.272374 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba72162-51d7-45dd-bf87-a18b04632fb2 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:42.273860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 500/1018] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:42.279294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417da5fa-a417-4070-af58-ec9fa875a409 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:42.279577 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417da5fa-a417-4070-af58-ec9fa875a409 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:42.279873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417da5fa-a417-4070-af58-ec9fa875a409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:42.298658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-417da5fa-a417-4070-af58-ec9fa875a409 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:42.299884 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-417da5fa-a417-4070-af58-ec9fa875a409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:42.378004 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 519/1019] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:42 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:42.400990 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba72162-51d7-45dd-bf87-a18b04632fb2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:42.512512 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba72162-51d7-45dd-bf87-a18b04632fb2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:42.537908 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:42.552199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32945be3-c24a-46e7-b314-2a30864f1660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:42.552894 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32945be3-c24a-46e7-b314-2a30864f1660 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:42.553177 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32945be3-c24a-46e7-b314-2a30864f1660 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:42.553445 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32945be3-c24a-46e7-b314-2a30864f1660 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:42.565143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32945be3-c24a-46e7-b314-2a30864f1660 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:47:42.570206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-32945be3-c24a-46e7-b314-2a30864f1660 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 04:47:42.570206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:42.570206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:42.570206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:42.570206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:42.570206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:42.570206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:42.570206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:42.570206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:42.570206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:42.570206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:42.570206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:42.570206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:47:42.570206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:47:42.570206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:47:42.570206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:47:42.571200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:47:42.571200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:47:42.571200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:47:42.571200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:47:42.571200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:42.571200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:42.571200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:42.571200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:42.571200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:42.571200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:42.571200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:42.571200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:42.571200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:42.571200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:42.571200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:42.571200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:42.576376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:42.576376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:42.576376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:42.576376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:42.576376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:47:42.576376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:47:42.576376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:47:42.576376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:47:42.576376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:47:42.576376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:47:42.576376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 04:47:42.576376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:42.576376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 501/1020] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Jul 26 04:47:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:42.579533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417da5fa-a417-4070-af58-ec9fa875a409 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:42.588275 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a849f8a-1f07-4225-a0ec-70315ea317bc None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:42.589481 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a849f8a-1f07-4225-a0ec-70315ea317bc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:42.589800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a849f8a-1f07-4225-a0ec-70315ea317bc None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:42.591597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 502/1021] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:47:42.597323 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03e89c-6f3c-4f76-a51d-218c21a7a93e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:42.597635 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03e89c-6f3c-4f76-a51d-218c21a7a93e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:42.598058 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03e89c-6f3c-4f76-a51d-218c21a7a93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:42.616427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8c03e89c-6f3c-4f76-a51d-218c21a7a93e None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:42.617652 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8c03e89c-6f3c-4f76-a51d-218c21a7a93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:42.624102 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:42.694554 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 503/1022] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:42 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:42.701821 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417da5fa-a417-4070-af58-ec9fa875a409 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:42.717425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03e89c-6f3c-4f76-a51d-218c21a7a93e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:42.737136 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b568bc8-4131-4dde-b8d0-7db6e8e25934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:42.737842 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b568bc8-4131-4dde-b8d0-7db6e8e25934 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:42.738098 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b568bc8-4131-4dde-b8d0-7db6e8e25934 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:42.738373 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b568bc8-4131-4dde-b8d0-7db6e8e25934 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:42.739158 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b568bc8-4131-4dde-b8d0-7db6e8e25934 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:42.739665 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b568bc8-4131-4dde-b8d0-7db6e8e25934 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:42.741077 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b568bc8-4131-4dde-b8d0-7db6e8e25934 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:42.750280 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 520/1023] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Jul 26 04:47:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:42.755969 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b568bc8-4131-4dde-b8d0-7db6e8e25934 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:42.829439 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03e89c-6f3c-4f76-a51d-218c21a7a93e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:42.844042 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:42.863572 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:42.867997 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc370488-7e89-4bd9-8692-e1b2e5c71914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:42.868439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b568bc8-4131-4dde-b8d0-7db6e8e25934 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:42.868870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc370488-7e89-4bd9-8692-e1b2e5c71914 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:42.869158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc370488-7e89-4bd9-8692-e1b2e5c71914 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:42.869438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc370488-7e89-4bd9-8692-e1b2e5c71914 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:42.870782 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc370488-7e89-4bd9-8692-e1b2e5c71914 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:42.873222 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc370488-7e89-4bd9-8692-e1b2e5c71914 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:42.892673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 504/1024] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jul 26 04:47:42 2022] POST /identity/v3/services => generated 242 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:47:42.925749 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb5456-f428-4966-83dd-078513acf861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:42.926413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb5456-f428-4966-83dd-078513acf861 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:42.926692 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb5456-f428-4966-83dd-078513acf861 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:42.926960 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb5456-f428-4966-83dd-078513acf861 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:42.940460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacb5456-f428-4966-83dd-078513acf861 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:42.941962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacb5456-f428-4966-83dd-078513acf861 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:42.959814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-cacb5456-f428-4966-83dd-078513acf861 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:42.959814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:42.959814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:42.959814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:42.959814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:42.959814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:42.959814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:42.959814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:42.959814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:42.959814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:42.959814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:42.959814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:42.959814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:42.959814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:42.959814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:42.960993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:42.960993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:42.960993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:42.960993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:42.960993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:42.960993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:42.960993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:42.960993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:42.960993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:42.960993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:42.960993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:42.960993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:42.960993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:42.960993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:42.960993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:42.960993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:42.960993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:42.967852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:42.967852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:42.967852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:42.967852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:42.967852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 521/1025] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:47:42.968580 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc370488-7e89-4bd9-8692-e1b2e5c71914 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:43.079242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc370488-7e89-4bd9-8692-e1b2e5c71914 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:43.115261 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c5e29-c3d9-4368-938b-7a40e824f08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:43.115864 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c5e29-c3d9-4368-938b-7a40e824f08e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:43.116150 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c5e29-c3d9-4368-938b-7a40e824f08e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:43.116415 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c5e29-c3d9-4368-938b-7a40e824f08e None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:43.117119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4c5e29-c3d9-4368-938b-7a40e824f08e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:43.117421 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4c5e29-c3d9-4368-938b-7a40e824f08e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:43.118847 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4c5e29-c3d9-4368-938b-7a40e824f08e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:43.131978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 505/1026] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:43.137625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb5456-f428-4966-83dd-078513acf861 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:43.242615 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb5456-f428-4966-83dd-078513acf861 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:43.277878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d352ec28-95a9-443b-9032-b225ed0e8113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:43.278547 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352ec28-95a9-443b-9032-b225ed0e8113 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:43.278830 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352ec28-95a9-443b-9032-b225ed0e8113 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:43.279199 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352ec28-95a9-443b-9032-b225ed0e8113 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:43.288159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d352ec28-95a9-443b-9032-b225ed0e8113 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:43.289704 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d352ec28-95a9-443b-9032-b225ed0e8113 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:43.302220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-d352ec28-95a9-443b-9032-b225ed0e8113 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 04:47:43.302220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:43.302220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:43.302220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:43.302220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:43.302220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:43.302220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:43.302220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:43.302220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:43.302220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:43.302220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:43.302220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:43.302220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:47:43.302220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:47:43.302220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:47:43.302220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:47:43.303304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:43.303304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:43.303304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:47:43.303304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:43.303304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:47:43.303304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:43.303304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:43.303304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:43.303304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:43.303304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:43.303304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:43.303304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:43.303304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:43.303304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:43.303304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:43.303304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:43.303304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:43.303304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:43.309564 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:43.309564 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:43.309564 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:47:43.309564 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:47:43.309564 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:47:43.309564 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:47:43.309564 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:47:43.309564 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:47:43.309564 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 04:47:43.309564 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:43.309564 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 522/1027] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Jul 26 04:47:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:43.310189 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c5e29-c3d9-4368-938b-7a40e824f08e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:43.345175 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:43.423942 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c5e29-c3d9-4368-938b-7a40e824f08e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:43.441931 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f42c8-49d8-480c-b763-236f3257e859 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:43.442248 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f42c8-49d8-480c-b763-236f3257e859 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:43.442461 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f42c8-49d8-480c-b763-236f3257e859 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:43.444030 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 523/1028] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:47:43.449150 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b8a79-4c2f-4304-bb96-d84dec2751cd None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:43.449440 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b8a79-4c2f-4304-bb96-d84dec2751cd None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:43.449719 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b8a79-4c2f-4304-bb96-d84dec2751cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:43.462479 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395347b2-f968-4e51-bb2c-f34a27ff892b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:43.463630 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395347b2-f968-4e51-bb2c-f34a27ff892b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:43.464274 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395347b2-f968-4e51-bb2c-f34a27ff892b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:43.464593 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395347b2-f968-4e51-bb2c-f34a27ff892b None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:43.465643 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395347b2-f968-4e51-bb2c-f34a27ff892b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:43.465953 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395347b2-f968-4e51-bb2c-f34a27ff892b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:43.467406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395347b2-f968-4e51-bb2c-f34a27ff892b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:43.468904 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-526b8a79-4c2f-4304-bb96-d84dec2751cd None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:43.470155 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-526b8a79-4c2f-4304-bb96-d84dec2751cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:43.472275 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:43.491984 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 506/1029] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Jul 26 04:47:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:43.496445 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395347b2-f968-4e51-bb2c-f34a27ff892b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:43.543611 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 524/1030] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 04:47:43.565377 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b8a79-4c2f-4304-bb96-d84dec2751cd None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:43.597961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:43.607936 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395347b2-f968-4e51-bb2c-f34a27ff892b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:43.643956 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90eec4-d8d8-48ef-976a-8670a1b9c95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:43.644515 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90eec4-d8d8-48ef-976a-8670a1b9c95a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:43.644799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90eec4-d8d8-48ef-976a-8670a1b9c95a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:43.645072 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90eec4-d8d8-48ef-976a-8670a1b9c95a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:43.685169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b8a79-4c2f-4304-bb96-d84dec2751cd None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:43.723572 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf90eec4-d8d8-48ef-976a-8670a1b9c95a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:43.725858 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf90eec4-d8d8-48ef-976a-8670a1b9c95a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:43.727433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c329391-a4e5-4956-b180-4fb5ba22a0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:43.728521 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c329391-a4e5-4956-b180-4fb5ba22a0f9 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:43.729097 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c329391-a4e5-4956-b180-4fb5ba22a0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:43.729550 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c329391-a4e5-4956-b180-4fb5ba22a0f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:43.731066 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c329391-a4e5-4956-b180-4fb5ba22a0f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:43.733600 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c329391-a4e5-4956-b180-4fb5ba22a0f9 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:43.739583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 525/1031] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:43.742851 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c329391-a4e5-4956-b180-4fb5ba22a0f9 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:43.751708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:43.835202 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:43.851036 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 507/1032] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:47:43 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 04:47:43.851508 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c329391-a4e5-4956-b180-4fb5ba22a0f9 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:43.851792 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90eec4-d8d8-48ef-976a-8670a1b9c95a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:43.889494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671a52e-d65b-4e3d-a709-a055199ec6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:43.890123 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671a52e-d65b-4e3d-a709-a055199ec6ec None admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:43.890409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671a52e-d65b-4e3d-a709-a055199ec6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:43.890681 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671a52e-d65b-4e3d-a709-a055199ec6ec None admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:43.891374 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671a52e-d65b-4e3d-a709-a055199ec6ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:43.893269 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671a52e-d65b-4e3d-a709-a055199ec6ec None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:43.923831 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 526/1033] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jul 26 04:47:43 2022] POST /identity/v3/users => generated 285 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:47:43.965165 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90eec4-d8d8-48ef-976a-8670a1b9c95a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:44.004870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5671be-4c93-44f8-b7e0-42a9bc21b842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:44.005553 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5671be-4c93-44f8-b7e0-42a9bc21b842 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:44.005835 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5671be-4c93-44f8-b7e0-42a9bc21b842 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:44.006108 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5671be-4c93-44f8-b7e0-42a9bc21b842 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:44.019403 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5671be-4c93-44f8-b7e0-42a9bc21b842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:44.021044 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5671be-4c93-44f8-b7e0-42a9bc21b842 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:44.037549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-df5671be-4c93-44f8-b7e0-42a9bc21b842 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:44.037549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:44.037549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:44.037549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:44.037549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:44.037549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:44.037549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:44.037549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:44.037549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:44.037549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:44.037549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:44.037549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:44.037549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:44.037549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:44.037549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:44.038586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:44.038586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:44.038586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:44.038586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:44.038586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:44.038586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:44.038586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:44.038586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:44.038586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:44.038586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:44.038586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:44.038586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:44.038586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:44.038586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:44.038586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:44.038586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:44.038586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:44.039527 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:44.039527 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:44.039527 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:44.039527 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:44.045564 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 508/1034] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:44.045799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671a52e-d65b-4e3d-a709-a055199ec6ec None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:44.119043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:44.162624 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671a52e-d65b-4e3d-a709-a055199ec6ec None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:44.197397 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c99cc0-4d8f-4e17-9330-2d5595f9e866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:44.198851 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c99cc0-4d8f-4e17-9330-2d5595f9e866 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:44.199135 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c99cc0-4d8f-4e17-9330-2d5595f9e866 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:44.199407 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c99cc0-4d8f-4e17-9330-2d5595f9e866 None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:44.200148 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c99cc0-4d8f-4e17-9330-2d5595f9e866 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:44.200458 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c99cc0-4d8f-4e17-9330-2d5595f9e866 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:44.201865 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c99cc0-4d8f-4e17-9330-2d5595f9e866 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:44.215255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 527/1035] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:44.220931 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5671be-4c93-44f8-b7e0-42a9bc21b842 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:44.270666 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:44.328962 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5671be-4c93-44f8-b7e0-42a9bc21b842 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:44.366035 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec6008-00eb-45e9-a3e2-3b2534fcf24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:44.366689 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec6008-00eb-45e9-a3e2-3b2534fcf24e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:44.366966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec6008-00eb-45e9-a3e2-3b2534fcf24e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:44.367223 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec6008-00eb-45e9-a3e2-3b2534fcf24e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:44.374478 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec6008-00eb-45e9-a3e2-3b2534fcf24e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:44.375955 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec6008-00eb-45e9-a3e2-3b2534fcf24e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:44.387991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-09ec6008-00eb-45e9-a3e2-3b2534fcf24e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:44.387991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:44.387991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:44.387991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:44.387991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:44.387991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:44.387991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:44.387991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:44.387991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:44.387991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:44.387991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:44.387991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:44.387991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:47:44.387991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:47:44.387991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:47:44.388952 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:47:44.388952 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:44.388952 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:44.388952 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:44.388952 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:44.388952 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:44.388952 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:44.388952 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:44.388952 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:44.388952 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:44.388952 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:44.388952 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:44.388952 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:44.388952 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:44.388952 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:44.388952 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:44.388952 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:44.394823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:47:44.394823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:47:44.394823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:47:44.394823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:47:44.394823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:47:44.394823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:47:44.394823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:44.394823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:44.394823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 509/1036] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:47:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:44.398437 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c99cc0-4d8f-4e17-9330-2d5595f9e866 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:44.504109 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c99cc0-4d8f-4e17-9330-2d5595f9e866 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:44.528026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:44.541106 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9cdf74-5616-4ddc-9190-bdd9d9ec6e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:44.541751 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cdf74-5616-4ddc-9190-bdd9d9ec6e82 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:44.542038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cdf74-5616-4ddc-9190-bdd9d9ec6e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:44.542316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cdf74-5616-4ddc-9190-bdd9d9ec6e82 None admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:44.543081 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9cdf74-5616-4ddc-9190-bdd9d9ec6e82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:44.543398 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9cdf74-5616-4ddc-9190-bdd9d9ec6e82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:44.544881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9cdf74-5616-4ddc-9190-bdd9d9ec6e82 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:44.557254 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 528/1037] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:47:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:44.564034 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec6008-00eb-45e9-a3e2-3b2534fcf24e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:44.676589 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec6008-00eb-45e9-a3e2-3b2534fcf24e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:44.679116 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:44.714669 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37edb6-c14b-4faf-bd38-bb0a5d7e10bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:44.715300 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37edb6-c14b-4faf-bd38-bb0a5d7e10bf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:44.715647 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37edb6-c14b-4faf-bd38-bb0a5d7e10bf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:44.715940 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37edb6-c14b-4faf-bd38-bb0a5d7e10bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:44.716736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a37edb6-c14b-4faf-bd38-bb0a5d7e10bf None admin] RBAC: Adding query filter params (role.id=29516d985a494f4b88d90e1e4f525c24, scope.project.id=d583e0be8ef74766bffd6c3c07af18f1, user.id=2faeed9002ee4154bc980c146cd06bcc) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:44.717039 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a37edb6-c14b-4faf-bd38-bb0a5d7e10bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:44.718467 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a37edb6-c14b-4faf-bd38-bb0a5d7e10bf None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:44.724667 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 510/1038] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:47:44 2022] GET /identity/v3/role_assignments?user.id=2faeed9002ee4154bc980c146cd06bcc&scope.project.id=d583e0be8ef74766bffd6c3c07af18f1&role.id=29516d985a494f4b88d90e1e4f525c24 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:44.955739 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74fb67-4427-49d7-9c8f-235b6bb711f2 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:44.956323 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74fb67-4427-49d7-9c8f-235b6bb711f2 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:44.956323 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74fb67-4427-49d7-9c8f-235b6bb711f2 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:44.957866 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 529/1039] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:44.963242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa50fb-753f-4c52-aac9-68813b8af71b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:44.963531 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa50fb-753f-4c52-aac9-68813b8af71b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:44.964223 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa50fb-753f-4c52-aac9-68813b8af71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:44.983507 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-46aa50fb-753f-4c52-aac9-68813b8af71b None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:44.984745 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-46aa50fb-753f-4c52-aac9-68813b8af71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:45.058139 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 511/1040] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:45.080132 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74fb67-4427-49d7-9c8f-235b6bb711f2 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:45.190028 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74fb67-4427-49d7-9c8f-235b6bb711f2 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:45.225208 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70245961-c074-437f-9ea5-af72a77fa60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:45.225780 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70245961-c074-437f-9ea5-af72a77fa60d None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:45.226066 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70245961-c074-437f-9ea5-af72a77fa60d None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:45.226341 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70245961-c074-437f-9ea5-af72a77fa60d None admin] PATH_INFO: `/v3/services/image` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:45.227002 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70245961-c074-437f-9ea5-af72a77fa60d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:45.228505 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70245961-c074-437f-9ea5-af72a77fa60d None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:45.238053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-70245961-c074-437f-9ea5-af72a77fa60d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 04:47:45.238053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:45.238053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 04:47:45.238053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 04:47:45.238053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 04:47:45.238053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 04:47:45.238053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 04:47:45.238053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:45.238053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 04:47:45.238053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:45.238053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:47:45.238053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:45.238053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:45.238053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:45.238053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:45.238053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:45.238053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:45.239169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:45.239169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:45.239169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:45.239169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:45.239169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:45.239169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:45.239169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 04:47:45.239169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 04:47:45.239169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 04:47:45.239169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 04:47:45.239169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:45.239169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:45.239169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:45.239169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:45.239169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:45.239169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:45.240178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:45.240178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:45.240178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:45.240178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:45.240178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:45.240178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:45.240178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:45.240178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:45.240178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:45.240178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:45.240178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 04:47:45.240178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:45.240178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 04:47:45.240178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:45.241919 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 530/1041] 10.222.0.17 () {56 vars in 1105 bytes} [Tue Jul 26 04:47:45 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:45.247689 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa50fb-753f-4c52-aac9-68813b8af71b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:45.360939 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:45.365431 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa50fb-753f-4c52-aac9-68813b8af71b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:45.406319 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d9d00-9d6d-4f78-9abb-d32913098596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:45.406959 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d9d00-9d6d-4f78-9abb-d32913098596 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:45.407241 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d9d00-9d6d-4f78-9abb-d32913098596 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:45.407543 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d9d00-9d6d-4f78-9abb-d32913098596 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:45.408230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591d9d00-9d6d-4f78-9abb-d32913098596 None admin] RBAC: Adding query filter params (name=image) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:45.408537 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591d9d00-9d6d-4f78-9abb-d32913098596 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:45.409918 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591d9d00-9d6d-4f78-9abb-d32913098596 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:45.415846 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 512/1042] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:45.421413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70245961-c074-437f-9ea5-af72a77fa60d None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:45.472068 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:45.529415 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70245961-c074-437f-9ea5-af72a77fa60d None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:45.565136 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872462a9-a400-4ea4-a71d-8a72285a906c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:45.565840 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872462a9-a400-4ea4-a71d-8a72285a906c None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:45.566123 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872462a9-a400-4ea4-a71d-8a72285a906c None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:45.566396 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872462a9-a400-4ea4-a71d-8a72285a906c None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:45.567087 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872462a9-a400-4ea4-a71d-8a72285a906c None admin] RBAC: Adding query filter params (type=image) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:45.567395 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872462a9-a400-4ea4-a71d-8a72285a906c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:45.568840 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872462a9-a400-4ea4-a71d-8a72285a906c None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:45.579088 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 531/1043] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:45.580762 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d9d00-9d6d-4f78-9abb-d32913098596 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:45.689713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d9d00-9d6d-4f78-9abb-d32913098596 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:45.728869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400cbd75-f141-4274-a831-fe866a3587e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:45.729881 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cbd75-f141-4274-a831-fe866a3587e8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:45.730370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cbd75-f141-4274-a831-fe866a3587e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:45.730867 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cbd75-f141-4274-a831-fe866a3587e8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:45.731863 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400cbd75-f141-4274-a831-fe866a3587e8 None admin] RBAC: Adding query filter params (interface=public, service_id=257dbb2aae734e5594eed275e92c478a, region_id=RegionOne) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:45.732401 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400cbd75-f141-4274-a831-fe866a3587e8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:45.733981 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400cbd75-f141-4274-a831-fe866a3587e8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:45.740448 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 513/1044] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Jul 26 04:47:45 2022] GET /identity/v3/endpoints?service_id=257dbb2aae734e5594eed275e92c478a&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:45.746491 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872462a9-a400-4ea4-a71d-8a72285a906c None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:45.856859 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872462a9-a400-4ea4-a71d-8a72285a906c None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:45.867527 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:45.893149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e9544f-3865-4c37-be2c-1b1718a40b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:45.893777 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e9544f-3865-4c37-be2c-1b1718a40b59 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:45.894043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e9544f-3865-4c37-be2c-1b1718a40b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:45.894315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e9544f-3865-4c37-be2c-1b1718a40b59 None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:45.894983 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e9544f-3865-4c37-be2c-1b1718a40b59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:45.896392 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e9544f-3865-4c37-be2c-1b1718a40b59 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:45.903391 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 532/1045] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Jul 26 04:47:45 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 04:47:45.985246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1f496-0f05-4391-a1d3-d76b3a8681a6 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:45.985598 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1f496-0f05-4391-a1d3-d76b3a8681a6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:45.985801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1f496-0f05-4391-a1d3-d76b3a8681a6 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:45.987297 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 514/1046] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:45.993744 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784c45f-a107-4f00-b7bd-a90cfb926c88 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:45.994285 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784c45f-a107-4f00-b7bd-a90cfb926c88 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:45.994285 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784c45f-a107-4f00-b7bd-a90cfb926c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:46.014516 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-a784c45f-a107-4f00-b7bd-a90cfb926c88 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:46.015752 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a784c45f-a107-4f00-b7bd-a90cfb926c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:46.096180 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 533/1047] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:46.134913 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1f496-0f05-4391-a1d3-d76b3a8681a6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:46.176036 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:46.262015 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1f496-0f05-4391-a1d3-d76b3a8681a6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:46.299470 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354873d-5837-4bbb-972c-2a121d8f812e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:46.300172 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354873d-5837-4bbb-972c-2a121d8f812e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:46.300460 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354873d-5837-4bbb-972c-2a121d8f812e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:46.300731 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354873d-5837-4bbb-972c-2a121d8f812e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:46.312525 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2354873d-5837-4bbb-972c-2a121d8f812e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:47:46.317427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-2354873d-5837-4bbb-972c-2a121d8f812e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:47:46.317427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:46.317427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:46.317427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:46.317427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:46.317427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:46.317427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:46.317427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:46.317427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:46.317427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:46.317427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:46.317427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:46.317427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:47:46.317427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:47:46.317427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:47:46.317427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:47:46.318396 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:47:46.318396 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:47:46.318396 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:47:46.318396 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:47:46.318396 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:46.318396 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:46.318396 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:46.318396 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:46.318396 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:46.318396 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:46.318396 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:46.318396 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:46.318396 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:46.318396 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:46.318396 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:46.318396 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:46.324464 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:46.324464 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:46.324464 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:46.324464 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:46.324464 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:47:46.324464 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:47:46.324464 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:47:46.324464 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:47:46.324464 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:47:46.324464 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:47:46.324464 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:47:46.324464 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:46.324464 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 515/1048] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:47:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:46.326725 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784c45f-a107-4f00-b7bd-a90cfb926c88 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:46.439074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784c45f-a107-4f00-b7bd-a90cfb926c88 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:46.441506 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:46.477497 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba255b-94f4-4a36-be96-e844e149fae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:46.478131 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba255b-94f4-4a36-be96-e844e149fae0 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:46.478414 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba255b-94f4-4a36-be96-e844e149fae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:46.478696 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba255b-94f4-4a36-be96-e844e149fae0 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:46.479423 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ba255b-94f4-4a36-be96-e844e149fae0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:46.479765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ba255b-94f4-4a36-be96-e844e149fae0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:46.481166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ba255b-94f4-4a36-be96-e844e149fae0 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:46.494940 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 534/1049] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:47:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:46.496730 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354873d-5837-4bbb-972c-2a121d8f812e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:46.590849 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:46.607242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354873d-5837-4bbb-972c-2a121d8f812e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:46.644040 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3f8dd-6680-442d-b0ba-8c4da6248938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:46.644614 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3f8dd-6680-442d-b0ba-8c4da6248938 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:46.644896 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3f8dd-6680-442d-b0ba-8c4da6248938 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:46.645171 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3f8dd-6680-442d-b0ba-8c4da6248938 None admin] PATH_INFO: `/v3/users/7a2fbade789949fc826de29c4f997f41` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:46.660664 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d3f8dd-6680-442d-b0ba-8c4da6248938 None admin] RBAC: Authorizing `identity:get_user(user_id=7a2fbade789949fc826de29c4f997f41)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:46.662134 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d3f8dd-6680-442d-b0ba-8c4da6248938 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:46.664086 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 516/1050] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:47:46 2022] GET /identity/v3/users/7a2fbade789949fc826de29c4f997f41 => generated 285 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:46.669998 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba255b-94f4-4a36-be96-e844e149fae0 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:46.679992 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61ea39-02a4-45a9-a978-b5ba5a6603ba None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:46.680364 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61ea39-02a4-45a9-a978-b5ba5a6603ba None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:46.680645 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61ea39-02a4-45a9-a978-b5ba5a6603ba None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:46.682526 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 517/1051] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:47:46.687379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c54c9-81d7-4c91-8733-8ecf76db83c5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:46.687686 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c54c9-81d7-4c91-8733-8ecf76db83c5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:46.687955 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c54c9-81d7-4c91-8733-8ecf76db83c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:46.707049 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-867c54c9-81d7-4c91-8733-8ecf76db83c5 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:46.708297 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-867c54c9-81d7-4c91-8733-8ecf76db83c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:46.781027 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba255b-94f4-4a36-be96-e844e149fae0 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:46.786444 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 518/1052] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 04:47:46.808944 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c54c9-81d7-4c91-8733-8ecf76db83c5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:46.817417 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c78264-b4af-4faf-96c7-bce6f6a530f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:46.818138 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c78264-b4af-4faf-96c7-bce6f6a530f8 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:46.818346 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c78264-b4af-4faf-96c7-bce6f6a530f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:46.818613 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c78264-b4af-4faf-96c7-bce6f6a530f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:46.860034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:46.896060 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c78264-b4af-4faf-96c7-bce6f6a530f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:46.898838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c78264-b4af-4faf-96c7-bce6f6a530f8 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:46.901909 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:46.930866 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c54c9-81d7-4c91-8733-8ecf76db83c5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:46.971893 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4dc4b-4f95-4b30-9142-3349ceb31be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:46.972553 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4dc4b-4f95-4b30-9142-3349ceb31be7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:46.972909 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4dc4b-4f95-4b30-9142-3349ceb31be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:46.973422 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4dc4b-4f95-4b30-9142-3349ceb31be7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:46.987353 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4dc4b-4f95-4b30-9142-3349ceb31be7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:47:46.992805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d7e4dc4b-4f95-4b30-9142-3349ceb31be7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 04:47:46.992805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:46.992805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:46.992805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:46.992805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:46.992805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:46.992805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:46.992805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:46.992805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:46.992805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:46.992805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:46.992805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:46.992805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:47:46.992805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:47:46.992805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:47:46.992805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:47:46.993773 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:47:46.993773 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:47:46.993773 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:47:46.993773 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:47:46.993773 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:46.993773 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:46.993773 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:46.993773 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:46.993773 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:46.993773 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:46.993773 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:46.993773 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:46.993773 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:46.993773 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:46.993773 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:46.993773 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:46.994627 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:46.994627 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:46.994627 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:46.994627 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:46.994627 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:47:46.994627 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:47:46.994627 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:47:46.994627 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:47:46.994627 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:47:46.994627 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:47:46.994627 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 04:47:46.994627 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:46.994627 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 519/1053] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Jul 26 04:47:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:47.000139 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4dc4b-4f95-4b30-9142-3349ceb31be7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:47.014425 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 535/1054] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:47:46 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 04:47:47.020867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c78264-b4af-4faf-96c7-bce6f6a530f8 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:47.104222 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:47.135322 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4dc4b-4f95-4b30-9142-3349ceb31be7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:47.141878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c78264-b4af-4faf-96c7-bce6f6a530f8 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:47.171048 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d3872-3be9-430f-87e7-56dc708f65fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:47.171615 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d3872-3be9-430f-87e7-56dc708f65fb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:47.171904 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d3872-3be9-430f-87e7-56dc708f65fb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:47.172177 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d3872-3be9-430f-87e7-56dc708f65fb None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:47.172893 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d3872-3be9-430f-87e7-56dc708f65fb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:47.173191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d3872-3be9-430f-87e7-56dc708f65fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:47.174602 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d3872-3be9-430f-87e7-56dc708f65fb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:47.176087 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:47.176971 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76b705-6223-4591-9f5b-bba360cb9a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:47.177715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76b705-6223-4591-9f5b-bba360cb9a28 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:47.178042 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76b705-6223-4591-9f5b-bba360cb9a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:47.178362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76b705-6223-4591-9f5b-bba360cb9a28 None admin] PATH_INFO: `/v3/projects/2f7110dfeaa74ec1aecff6d2cad32543` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:47.179583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a76b705-6223-4591-9f5b-bba360cb9a28 None admin] RBAC: Authorizing `identity:get_project(project_id=2f7110dfeaa74ec1aecff6d2cad32543)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:47.181096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a76b705-6223-4591-9f5b-bba360cb9a28 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:47.182810 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 536/1055] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:47:47 2022] GET /identity/v3/projects/2f7110dfeaa74ec1aecff6d2cad32543 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:47.191141 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76b705-6223-4591-9f5b-bba360cb9a28 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:47.191613 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 520/1056] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Jul 26 04:47:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:47.196294 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d3872-3be9-430f-87e7-56dc708f65fb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:47.296590 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76b705-6223-4591-9f5b-bba360cb9a28 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:47.305000 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d3872-3be9-430f-87e7-56dc708f65fb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:47.331454 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c772cdf9-9132-4105-84b9-279b43a7602a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:47.332107 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772cdf9-9132-4105-84b9-279b43a7602a None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:47.332397 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772cdf9-9132-4105-84b9-279b43a7602a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:47.332676 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772cdf9-9132-4105-84b9-279b43a7602a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:47.333473 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c772cdf9-9132-4105-84b9-279b43a7602a None admin] RBAC: Adding query filter params (role.id=72bf8bd23d2c4b76ad32b7b35d6db412, scope.project.id=2f7110dfeaa74ec1aecff6d2cad32543, user.id=7a2fbade789949fc826de29c4f997f41) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:47.336797 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c772cdf9-9132-4105-84b9-279b43a7602a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:47.336797 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c772cdf9-9132-4105-84b9-279b43a7602a None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:47.337529 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:47.343323 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 537/1057] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:47:47 2022] GET /identity/v3/role_assignments?user.id=7a2fbade789949fc826de29c4f997f41&scope.project.id=2f7110dfeaa74ec1aecff6d2cad32543&role.id=72bf8bd23d2c4b76ad32b7b35d6db412 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:47.355903 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf4eb1-3298-4c56-88d0-57669b9cb3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:47.356590 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf4eb1-3298-4c56-88d0-57669b9cb3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:47.356875 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf4eb1-3298-4c56-88d0-57669b9cb3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:47.357155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf4eb1-3298-4c56-88d0-57669b9cb3f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:47.371381 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf4eb1-3298-4c56-88d0-57669b9cb3f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:47.373713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf4eb1-3298-4c56-88d0-57669b9cb3f4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:47.393783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-8eaf4eb1-3298-4c56-88d0-57669b9cb3f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:47.393783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:47.393783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:47.393783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:47.393783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:47.393783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:47.393783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:47.393783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:47.393783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:47.393783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:47.393783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:47.393783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:47.393783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:47.393783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:47.393783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:47.396232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:47.396232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:47.396232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:47.396232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:47.396232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:47.396232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:47.396232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:47.396232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:47.396232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:47.396232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:47.396232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:47.396232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:47.396232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:47.396232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:47.396232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:47.396232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:47.396232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:47.397161 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:47.397161 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:47.397161 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:47.397161 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:47.397880 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 521/1058] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:47:47.402057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c772cdf9-9132-4105-84b9-279b43a7602a None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:47.522863 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c772cdf9-9132-4105-84b9-279b43a7602a None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:47.558400 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047bbcee-137c-40f5-8d7a-1dfa30db334b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:47.559033 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bbcee-137c-40f5-8d7a-1dfa30db334b None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:47.559311 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bbcee-137c-40f5-8d7a-1dfa30db334b None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:47.559600 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bbcee-137c-40f5-8d7a-1dfa30db334b None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:47.560358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047bbcee-137c-40f5-8d7a-1dfa30db334b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:47.560667 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047bbcee-137c-40f5-8d7a-1dfa30db334b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:47.562073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047bbcee-137c-40f5-8d7a-1dfa30db334b None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:47.579480 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 538/1059] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:47.581041 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf4eb1-3298-4c56-88d0-57669b9cb3f4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:47.634801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:47.693321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf4eb1-3298-4c56-88d0-57669b9cb3f4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:47.732315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c9de3-abd3-4c3e-8835-4a02d018a4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:47.732973 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c9de3-abd3-4c3e-8835-4a02d018a4e3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:47.733263 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c9de3-abd3-4c3e-8835-4a02d018a4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:47.733537 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c9de3-abd3-4c3e-8835-4a02d018a4e3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:47.741833 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c9de3-abd3-4c3e-8835-4a02d018a4e3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:47.743277 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c9de3-abd3-4c3e-8835-4a02d018a4e3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:47.755182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a08c9de3-abd3-4c3e-8835-4a02d018a4e3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 04:47:47.755182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:47.755182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:47.755182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:47.755182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:47.755182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:47.755182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:47.755182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:47.755182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:47.755182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:47.755182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:47.755182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:47.755182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:47:47.755182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:47:47.755182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:47:47.755182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:47:47.756205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:47.756205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:47.756205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:47:47.756205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:47.756205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:47:47.756205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:47.756205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:47.756205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:47.756205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:47.756205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:47.756205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:47.756205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:47.756205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:47.756205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:47.756205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:47.756205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:47.756205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:47.756205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:47.757311 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:47.757311 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:47.757311 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:47:47.757311 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:47:47.757311 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:47:47.757311 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:47:47.757311 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:47:47.757311 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:47:47.757311 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 04:47:47.757311 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:47.757311 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 522/1060] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Jul 26 04:47:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:47.763796 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047bbcee-137c-40f5-8d7a-1dfa30db334b None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:47.828663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:47.873464 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047bbcee-137c-40f5-8d7a-1dfa30db334b None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:47.909828 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936998e5-8255-4044-a489-1feff6bdd9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:47.910458 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936998e5-8255-4044-a489-1feff6bdd9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:47.910740 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936998e5-8255-4044-a489-1feff6bdd9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:47.911016 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936998e5-8255-4044-a489-1feff6bdd9b2 None admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:47.911843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936998e5-8255-4044-a489-1feff6bdd9b2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:47.912151 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936998e5-8255-4044-a489-1feff6bdd9b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:47.913606 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936998e5-8255-4044-a489-1feff6bdd9b2 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:47.933593 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 539/1061] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Jul 26 04:47:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:47.939959 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c9de3-abd3-4c3e-8835-4a02d018a4e3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:47.948732 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f359b81-aeed-424d-93fc-0fe72266110e None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:47.949058 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f359b81-aeed-424d-93fc-0fe72266110e None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:47.949299 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f359b81-aeed-424d-93fc-0fe72266110e None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:47.950708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 540/1062] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:47.956306 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd507b-2df9-470f-9b84-53673a9a4939 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:47.956589 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd507b-2df9-470f-9b84-53673a9a4939 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:47.956880 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd507b-2df9-470f-9b84-53673a9a4939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:47.976467 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-06fd507b-2df9-470f-9b84-53673a9a4939 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:47.977662 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-06fd507b-2df9-470f-9b84-53673a9a4939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:48.053294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 541/1063] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:47 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 04:47:48.058837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c9de3-abd3-4c3e-8835-4a02d018a4e3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:48.069180 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:48.074836 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd507b-2df9-470f-9b84-53673a9a4939 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:48.098496 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b5dbca-70e2-4e6b-a52d-3fad113094a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:48.099081 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5dbca-70e2-4e6b-a52d-3fad113094a1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:48.099368 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5dbca-70e2-4e6b-a52d-3fad113094a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:48.099674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5dbca-70e2-4e6b-a52d-3fad113094a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:48.111027 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b5dbca-70e2-4e6b-a52d-3fad113094a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:48.112533 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b5dbca-70e2-4e6b-a52d-3fad113094a1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:48.127108 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c0b5dbca-70e2-4e6b-a52d-3fad113094a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:48.127108 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:48.127108 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:48.127108 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:48.127108 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:48.127108 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:48.127108 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:48.127108 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:48.127108 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:48.127108 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:48.127108 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:48.127108 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:48.127108 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:48.127108 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:48.127108 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:48.128200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:48.128200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:48.128200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:48.128200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:48.128200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:48.128200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:48.128200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:48.128200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:48.128200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:48.128200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:48.128200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:48.128200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:48.128200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:48.128200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:48.128200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:48.128200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:48.128200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:48.134161 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:48.134161 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:48.134161 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:48.134161 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:48.134161 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 523/1064] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:48.134722 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:48.136570 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b5dbca-70e2-4e6b-a52d-3fad113094a1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:48.193994 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd507b-2df9-470f-9b84-53673a9a4939 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:48.232859 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98869408-6eb6-4225-8ce6-5a9520c041a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:48.233527 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98869408-6eb6-4225-8ce6-5a9520c041a6 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:48.233820 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98869408-6eb6-4225-8ce6-5a9520c041a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:48.234097 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98869408-6eb6-4225-8ce6-5a9520c041a6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:48.234802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98869408-6eb6-4225-8ce6-5a9520c041a6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:48.236267 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98869408-6eb6-4225-8ce6-5a9520c041a6 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:48.241215 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b5dbca-70e2-4e6b-a52d-3fad113094a1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:48.245325 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-98869408-6eb6-4225-8ce6-5a9520c041a6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 04:47:48.245325 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:48.245325 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 04:47:48.245325 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 04:47:48.245325 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 04:47:48.245325 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 04:47:48.245325 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 04:47:48.245325 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:48.245325 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 04:47:48.245325 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:48.245325 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:47:48.245325 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:48.245325 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:48.245325 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:48.245325 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:48.245325 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:48.245325 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:48.246308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:48.246308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:48.246308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:48.246308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:48.246308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:48.246308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:48.246308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 04:47:48.246308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 04:47:48.246308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 04:47:48.246308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 04:47:48.246308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:48.246308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:48.246308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:48.246308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:48.246308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:48.246308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:48.247379 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:48.247379 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:48.247379 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:48.247379 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:48.247379 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:48.247379 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:48.247379 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:48.247379 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:48.247379 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:48.247379 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:48.247379 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 04:47:48.247379 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:47:48.247379 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 04:47:48.247379 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:48.247379 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 542/1065] 10.222.0.17 () {56 vars in 1105 bytes} [Tue Jul 26 04:47:48 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:47:48.252398 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98869408-6eb6-4225-8ce6-5a9520c041a6 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:48.286190 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2fc8cd-c295-4099-b1c6-9db34565c2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:48.286829 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2fc8cd-c295-4099-b1c6-9db34565c2ab None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:48.287113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2fc8cd-c295-4099-b1c6-9db34565c2ab None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:48.287422 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2fc8cd-c295-4099-b1c6-9db34565c2ab None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:48.288819 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2fc8cd-c295-4099-b1c6-9db34565c2ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:48.288819 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2fc8cd-c295-4099-b1c6-9db34565c2ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:48.289917 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2fc8cd-c295-4099-b1c6-9db34565c2ab None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:48.303191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 524/1066] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:48.309133 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2fc8cd-c295-4099-b1c6-9db34565c2ab None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:48.333497 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:48.363697 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98869408-6eb6-4225-8ce6-5a9520c041a6 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:48.375466 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:48.404238 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4cf51-a98b-4613-a200-cb2d77093fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:48.404901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4cf51-a98b-4613-a200-cb2d77093fff None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:48.405268 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4cf51-a98b-4613-a200-cb2d77093fff None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:48.405412 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4cf51-a98b-4613-a200-cb2d77093fff None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:48.406241 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4cf51-a98b-4613-a200-cb2d77093fff None admin] RBAC: Adding query filter params (name=image) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:48.406551 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4cf51-a98b-4613-a200-cb2d77093fff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:48.407983 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4cf51-a98b-4613-a200-cb2d77093fff None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:48.413203 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 543/1067] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:48.419038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4cf51-a98b-4613-a200-cb2d77093fff None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:48.433832 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2fc8cd-c295-4099-b1c6-9db34565c2ab None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:48.474297 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da6e29-1b16-4e67-a331-55d1424c00cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:48.474965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da6e29-1b16-4e67-a331-55d1424c00cf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:48.475245 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da6e29-1b16-4e67-a331-55d1424c00cf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:48.475708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da6e29-1b16-4e67-a331-55d1424c00cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:48.484490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da6e29-1b16-4e67-a331-55d1424c00cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:48.486102 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da6e29-1b16-4e67-a331-55d1424c00cf None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:48.499268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c8da6e29-1b16-4e67-a331-55d1424c00cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:48.499268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:48.499268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:48.499268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:48.499268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:48.499268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:48.499268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:48.499268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:48.499268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:48.499268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:48.499268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:48.499268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:48.499268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:47:48.499268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:47:48.499268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:47:48.500557 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:47:48.500557 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:48.500557 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:48.500557 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:48.500557 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:48.500557 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:48.500557 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:48.500557 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:48.500557 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:48.500557 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:48.500557 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:48.500557 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:48.500557 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:48.500557 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:48.500557 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:48.500557 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:48.500557 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:48.501668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:47:48.501668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:47:48.501668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:47:48.501668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:47:48.501668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:47:48.501668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:47:48.501668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:48.501668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:48.503145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 525/1068] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:47:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:47:48.509199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da6e29-1b16-4e67-a331-55d1424c00cf None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:48.531180 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4cf51-a98b-4613-a200-cb2d77093fff None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:48.575458 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b28a7-85fd-4761-ab5c-390fba2a9cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:48.576882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b28a7-85fd-4761-ab5c-390fba2a9cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:48.577375 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b28a7-85fd-4761-ab5c-390fba2a9cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:48.578050 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b28a7-85fd-4761-ab5c-390fba2a9cf8 None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:48.579062 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b28a7-85fd-4761-ab5c-390fba2a9cf8 None admin] RBAC: Adding query filter params (type=image) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:48.579529 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b28a7-85fd-4761-ab5c-390fba2a9cf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:48.581887 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b28a7-85fd-4761-ab5c-390fba2a9cf8 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:48.591744 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 544/1069] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:48 2022] GET /identity/v3/services?type=image => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:48.594487 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b28a7-85fd-4761-ab5c-390fba2a9cf8 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:48.596793 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:48.618715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da6e29-1b16-4e67-a331-55d1424c00cf None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:48.641567 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:48.655761 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0e76c-85fe-4230-8fd1-63294b974eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:48.656395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0e76c-85fe-4230-8fd1-63294b974eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:48.656681 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0e76c-85fe-4230-8fd1-63294b974eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:48.656957 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0e76c-85fe-4230-8fd1-63294b974eb5 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:48.657717 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff0e76c-85fe-4230-8fd1-63294b974eb5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:48.658024 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff0e76c-85fe-4230-8fd1-63294b974eb5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:48.659969 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff0e76c-85fe-4230-8fd1-63294b974eb5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:48.672402 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 526/1070] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:47:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:48.678851 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0e76c-85fe-4230-8fd1-63294b974eb5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:48.725836 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b28a7-85fd-4761-ab5c-390fba2a9cf8 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:48.769173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47405ae6-db01-4b27-b4d2-52963bcab91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:48.770451 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47405ae6-db01-4b27-b4d2-52963bcab91f None admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:48.770741 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47405ae6-db01-4b27-b4d2-52963bcab91f None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:48.771021 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47405ae6-db01-4b27-b4d2-52963bcab91f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:48.771783 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47405ae6-db01-4b27-b4d2-52963bcab91f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:48.773710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47405ae6-db01-4b27-b4d2-52963bcab91f None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:48.786148 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0e76c-85fe-4230-8fd1-63294b974eb5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:48.796368 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 545/1071] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 04:47:48 2022] POST /identity/v3/endpoints => generated 327 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 04:47:48.842261 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c1eb04-6480-48a6-aa6d-80ddfaa30624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:48.842964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1eb04-6480-48a6-aa6d-80ddfaa30624 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:48.843248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1eb04-6480-48a6-aa6d-80ddfaa30624 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:48.843611 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1eb04-6480-48a6-aa6d-80ddfaa30624 None admin] PATH_INFO: `/v3/projects/d583e0be8ef74766bffd6c3c07af18f1/users/2faeed9002ee4154bc980c146cd06bcc/roles/29516d985a494f4b88d90e1e4f525c24` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:48.845965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c1eb04-6480-48a6-aa6d-80ddfaa30624 None admin] RBAC: Authorizing `identity:create_grant(project_id=d583e0be8ef74766bffd6c3c07af18f1, user_id=2faeed9002ee4154bc980c146cd06bcc, role_id=29516d985a494f4b88d90e1e4f525c24)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:48.847695 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c1eb04-6480-48a6-aa6d-80ddfaa30624 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:48.852212 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:48.867541 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 527/1072] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jul 26 04:47:48 2022] PUT /identity/v3/projects/d583e0be8ef74766bffd6c3c07af18f1/users/2faeed9002ee4154bc980c146cd06bcc/roles/29516d985a494f4b88d90e1e4f525c24 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 04:47:49.346081 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d778107-4b14-4a39-9309-02b8c3aa9b1d None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:49.346610 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d778107-4b14-4a39-9309-02b8c3aa9b1d None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:49.346610 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d778107-4b14-4a39-9309-02b8c3aa9b1d None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:49.348082 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 546/1073] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:49.353700 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3829b65-8e08-4a71-8a14-81f318698895 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:49.353975 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3829b65-8e08-4a71-8a14-81f318698895 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:49.354258 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3829b65-8e08-4a71-8a14-81f318698895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:49.373376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d3829b65-8e08-4a71-8a14-81f318698895 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:49.374579 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d3829b65-8e08-4a71-8a14-81f318698895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:49.449095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 528/1074] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:49.471228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d778107-4b14-4a39-9309-02b8c3aa9b1d None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:49.574730 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:49.585729 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d778107-4b14-4a39-9309-02b8c3aa9b1d None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:49.623664 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cbc14-992f-495c-91b2-457640f4082a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:49.624326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cbc14-992f-495c-91b2-457640f4082a None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:49.624611 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cbc14-992f-495c-91b2-457640f4082a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:49.624886 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cbc14-992f-495c-91b2-457640f4082a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:49.635920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45cbc14-992f-495c-91b2-457640f4082a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:47:49.640790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-a45cbc14-992f-495c-91b2-457640f4082a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:47:49.640790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:49.640790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:49.640790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:49.640790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:49.640790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:49.640790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:49.640790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:49.640790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:49.640790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:49.640790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:49.640790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:49.640790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:47:49.640790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:47:49.640790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:47:49.640790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:47:49.642189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:47:49.642189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:47:49.642189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:47:49.642189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:47:49.642189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:49.642189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:49.642189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:49.642189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:49.642189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:49.642189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:49.642189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:49.642189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:49.642189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:49.642189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:49.642189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:49.642189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:49.643069 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:49.643069 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:49.643069 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:49.643069 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:49.643069 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:47:49.643069 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:47:49.643069 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:47:49.643069 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:47:49.643069 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:47:49.643069 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:47:49.643069 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:47:49.643069 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:49.643069 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 547/1075] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:47:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:49.648125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3829b65-8e08-4a71-8a14-81f318698895 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:49.754945 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3829b65-8e08-4a71-8a14-81f318698895 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:49.787635 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:49.798052 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1dcb8-247e-4ca1-926a-d9f3061902f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:49.798692 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1dcb8-247e-4ca1-926a-d9f3061902f2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:49.798976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1dcb8-247e-4ca1-926a-d9f3061902f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:49.799246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1dcb8-247e-4ca1-926a-d9f3061902f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:49.800002 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c1dcb8-247e-4ca1-926a-d9f3061902f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:49.800310 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c1dcb8-247e-4ca1-926a-d9f3061902f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:49.801697 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c1dcb8-247e-4ca1-926a-d9f3061902f2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:49.810901 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 529/1076] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:47:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:49.816671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cbc14-992f-495c-91b2-457640f4082a None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:49.924238 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cbc14-992f-495c-91b2-457640f4082a None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:49.967928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ee447-ac6a-42ac-9b5a-025210065cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:49.968595 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ee447-ac6a-42ac-9b5a-025210065cff None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:49.968882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ee447-ac6a-42ac-9b5a-025210065cff None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:49.969158 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ee447-ac6a-42ac-9b5a-025210065cff None admin] PATH_INFO: `/v3/users/7a2fbade789949fc826de29c4f997f41` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:49.970391 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ee447-ac6a-42ac-9b5a-025210065cff None admin] RBAC: Authorizing `identity:get_user(user_id=7a2fbade789949fc826de29c4f997f41)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:49.971900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ee447-ac6a-42ac-9b5a-025210065cff None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:49.973939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 548/1077] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:47:49 2022] GET /identity/v3/users/7a2fbade789949fc826de29c4f997f41 => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:49.979758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1dcb8-247e-4ca1-926a-d9f3061902f2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:50.090667 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1dcb8-247e-4ca1-926a-d9f3061902f2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:50.126872 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936a621-0273-4afb-9685-58416a521f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:50.127533 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936a621-0273-4afb-9685-58416a521f46 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:50.127863 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936a621-0273-4afb-9685-58416a521f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:50.128220 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936a621-0273-4afb-9685-58416a521f46 None admin] PATH_INFO: `/v3/projects/2f7110dfeaa74ec1aecff6d2cad32543` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:50.129331 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936a621-0273-4afb-9685-58416a521f46 None admin] RBAC: Authorizing `identity:get_project(project_id=2f7110dfeaa74ec1aecff6d2cad32543)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:50.130780 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936a621-0273-4afb-9685-58416a521f46 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:50.138093 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 530/1078] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:47:49 2022] GET /identity/v3/projects/2f7110dfeaa74ec1aecff6d2cad32543 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:50.139184 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ee447-ac6a-42ac-9b5a-025210065cff None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:50.164900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:50.251269 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ee447-ac6a-42ac-9b5a-025210065cff None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:50.285945 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f537cd0-e271-4c44-becc-a8faff0bb73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:50.286637 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f537cd0-e271-4c44-becc-a8faff0bb73a None admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:50.286918 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f537cd0-e271-4c44-becc-a8faff0bb73a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:50.287193 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f537cd0-e271-4c44-becc-a8faff0bb73a None admin] PATH_INFO: `/v3/projects/2f7110dfeaa74ec1aecff6d2cad32543/users/7a2fbade789949fc826de29c4f997f41/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:50.289516 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f537cd0-e271-4c44-becc-a8faff0bb73a None admin] RBAC: Authorizing `identity:create_grant(project_id=2f7110dfeaa74ec1aecff6d2cad32543, user_id=7a2fbade789949fc826de29c4f997f41, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:50.291003 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f537cd0-e271-4c44-becc-a8faff0bb73a None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:50.311936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 549/1079] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jul 26 04:47:50 2022] PUT /identity/v3/projects/2f7110dfeaa74ec1aecff6d2cad32543/users/7a2fbade789949fc826de29c4f997f41/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 04:47:50.867707 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9b4a5-337c-42bb-8302-ec11bffa2939 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:50.868274 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9b4a5-337c-42bb-8302-ec11bffa2939 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:50.868274 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9b4a5-337c-42bb-8302-ec11bffa2939 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:50.870033 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 531/1080] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:50.876092 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6852cf-2a56-412c-9b56-b158e868c4f4 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:50.876636 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6852cf-2a56-412c-9b56-b158e868c4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:50.876636 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6852cf-2a56-412c-9b56-b158e868c4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:50.878802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0fa43-7f47-4ba2-abd0-1f820a64f0b5 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:50.879054 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0fa43-7f47-4ba2-abd0-1f820a64f0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:50.879329 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0fa43-7f47-4ba2-abd0-1f820a64f0b5 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:50.880737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 532/1081] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:47:50.886807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4bbde8-7504-47f5-a907-ad516bb4f9af None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:50.887119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4bbde8-7504-47f5-a907-ad516bb4f9af None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:50.887383 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4bbde8-7504-47f5-a907-ad516bb4f9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:50.895865 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-9a6852cf-2a56-412c-9b56-b158e868c4f4 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:50.897106 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-9a6852cf-2a56-412c-9b56-b158e868c4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:50.906913 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ea4bbde8-7504-47f5-a907-ad516bb4f9af None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:50.908492 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ea4bbde8-7504-47f5-a907-ad516bb4f9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:50.966144 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:50.982022 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 550/1082] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:50.987728 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 533/1083] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:51.004146 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6852cf-2a56-412c-9b56-b158e868c4f4 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:51.010539 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4bbde8-7504-47f5-a907-ad516bb4f9af None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:51.021651 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:51.122074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6852cf-2a56-412c-9b56-b158e868c4f4 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:51.129240 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4bbde8-7504-47f5-a907-ad516bb4f9af None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:51.158960 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418283f8-88ce-48ca-9e0d-a93e0b82d0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:51.159617 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418283f8-88ce-48ca-9e0d-a93e0b82d0ad None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:51.159902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418283f8-88ce-48ca-9e0d-a93e0b82d0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:51.160178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418283f8-88ce-48ca-9e0d-a93e0b82d0ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:51.167371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c2425-d00b-4a10-a7d5-e7d4430dacaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:51.168268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c2425-d00b-4a10-a7d5-e7d4430dacaf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:51.168719 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c2425-d00b-4a10-a7d5-e7d4430dacaf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:51.169231 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c2425-d00b-4a10-a7d5-e7d4430dacaf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:51.180519 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718c2425-d00b-4a10-a7d5-e7d4430dacaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:47:51.186074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-718c2425-d00b-4a10-a7d5-e7d4430dacaf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 04:47:51.186074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:51.186074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:51.186074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:51.186074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:51.186074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:51.186074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:51.186074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:51.186074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:51.186074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:51.186074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:51.186074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:51.186074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:47:51.186074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:47:51.186074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:47:51.186074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:47:51.187209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:47:51.187209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:47:51.187209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:47:51.187209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:47:51.187209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:51.187209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:51.187209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:51.187209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:51.187209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:51.187209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:51.187209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:51.187209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:51.187209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:51.187209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:51.187209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:51.187209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:51.188155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:51.188155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:51.188155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:51.188155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:51.188155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:47:51.188155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:47:51.188155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:47:51.188155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:47:51.188155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:47:51.188155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:47:51.188155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 04:47:51.188155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:51.189084 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 534/1084] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Jul 26 04:47:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:51.195054 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c2425-d00b-4a10-a7d5-e7d4430dacaf None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:51.237162 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418283f8-88ce-48ca-9e0d-a93e0b82d0ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:51.238655 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418283f8-88ce-48ca-9e0d-a93e0b82d0ad None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:51.248976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:51.254375 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:51.310405 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c2425-d00b-4a10-a7d5-e7d4430dacaf None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:51.352671 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f19bf44-2c98-4ad1-8f44-3c7a3c381496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:51.353309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19bf44-2c98-4ad1-8f44-3c7a3c381496 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:51.353582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19bf44-2c98-4ad1-8f44-3c7a3c381496 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:51.353860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19bf44-2c98-4ad1-8f44-3c7a3c381496 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:51.354581 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f19bf44-2c98-4ad1-8f44-3c7a3c381496 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:51.354884 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f19bf44-2c98-4ad1-8f44-3c7a3c381496 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:51.356296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f19bf44-2c98-4ad1-8f44-3c7a3c381496 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:51.358594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 551/1085] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:47:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 04:47:51.364586 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418283f8-88ce-48ca-9e0d-a93e0b82d0ad None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:51.372228 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 535/1086] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Jul 26 04:47:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:51.373433 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f19bf44-2c98-4ad1-8f44-3c7a3c381496 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:51.491612 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f19bf44-2c98-4ad1-8f44-3c7a3c381496 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:51.492011 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418283f8-88ce-48ca-9e0d-a93e0b82d0ad None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:51.502883 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:51.509760 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:51.530685 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223cb3ac-93a6-45bf-9a38-00561705575a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:51.531347 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cb3ac-93a6-45bf-9a38-00561705575a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:51.531661 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cb3ac-93a6-45bf-9a38-00561705575a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:51.531940 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cb3ac-93a6-45bf-9a38-00561705575a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:51.533182 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb3cba-522d-433d-b1ed-76c48b2310d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:51.533835 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb3cba-522d-433d-b1ed-76c48b2310d1 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:51.534119 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb3cba-522d-433d-b1ed-76c48b2310d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:51.534391 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb3cba-522d-433d-b1ed-76c48b2310d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:51.543974 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223cb3ac-93a6-45bf-9a38-00561705575a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:51.545445 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223cb3ac-93a6-45bf-9a38-00561705575a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:51.545553 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcb3cba-522d-433d-b1ed-76c48b2310d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:51.546905 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcb3cba-522d-433d-b1ed-76c48b2310d1 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:51.560787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-223cb3ac-93a6-45bf-9a38-00561705575a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:51.560787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:51.560787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:51.560787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:51.560787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:51.560787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:51.560787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:51.560787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:51.560787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:51.560787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:51.560787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:51.560787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:51.560787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:51.560787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:51.560787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:51.561736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:51.561736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:51.561736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:51.561736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:51.561736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:51.561736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:51.561736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:51.561736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:51.561736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:51.561736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:51.561736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:51.561736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:51.561736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:51.561736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:51.561736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:51.561736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:51.561736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:51.562857 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:51.562857 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:51.562857 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:51.562857 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:51.562857 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 536/1087] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:51.563878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-0dcb3cba-522d-433d-b1ed-76c48b2310d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:51.563878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:51.563878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:51.563878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:51.563878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:51.563878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:51.563878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:51.563878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:51.563878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:51.563878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:51.563878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:51.563878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:51.563878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:51.563878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:51.563878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:51.564755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:51.564755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:51.564755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:51.564755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:51.564755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:51.564755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:51.564755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:51.564755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:51.564755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:51.564755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:51.564755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:51.564755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:51.564755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:51.564755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:51.564755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:51.564755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:51.564755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:51.565672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:51.565672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:51.565672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:51.565672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:51.567725 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 552/1088] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:47:51.567835 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223cb3ac-93a6-45bf-9a38-00561705575a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:51.571319 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb3cba-522d-433d-b1ed-76c48b2310d1 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:51.676145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223cb3ac-93a6-45bf-9a38-00561705575a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:51.688223 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb3cba-522d-433d-b1ed-76c48b2310d1 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:51.711718 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde85245-ee9d-4ab1-82f8-ab1c21c47b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:51.712352 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde85245-ee9d-4ab1-82f8-ab1c21c47b81 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:51.712635 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde85245-ee9d-4ab1-82f8-ab1c21c47b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:51.712906 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde85245-ee9d-4ab1-82f8-ab1c21c47b81 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:51.714103 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde85245-ee9d-4ab1-82f8-ab1c21c47b81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:51.714383 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde85245-ee9d-4ab1-82f8-ab1c21c47b81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:51.715921 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde85245-ee9d-4ab1-82f8-ab1c21c47b81 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:51.725619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d906b-75c0-46e6-a75a-07c27f48d7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:51.726249 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d906b-75c0-46e6-a75a-07c27f48d7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:51.726560 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d906b-75c0-46e6-a75a-07c27f48d7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:51.726837 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d906b-75c0-46e6-a75a-07c27f48d7b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:51.727547 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d906b-75c0-46e6-a75a-07c27f48d7b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:51.727882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d906b-75c0-46e6-a75a-07c27f48d7b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:51.729269 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d906b-75c0-46e6-a75a-07c27f48d7b4 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:51.731113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 537/1089] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:51.736406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde85245-ee9d-4ab1-82f8-ab1c21c47b81 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:51.751627 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 553/1090] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:51.768831 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:51.865787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde85245-ee9d-4ab1-82f8-ab1c21c47b81 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:51.905086 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8256d8-0cac-4d58-9682-12f338717d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:51.905919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8256d8-0cac-4d58-9682-12f338717d68 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:51.906191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8256d8-0cac-4d58-9682-12f338717d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:51.906291 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8256d8-0cac-4d58-9682-12f338717d68 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:51.915501 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8256d8-0cac-4d58-9682-12f338717d68 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:51.916995 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8256d8-0cac-4d58-9682-12f338717d68 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:51.929806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-2c8256d8-0cac-4d58-9682-12f338717d68 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 04:47:51.929806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:51.929806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:51.929806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:51.929806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:51.929806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:51.929806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:51.929806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:51.929806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:51.929806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:51.929806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:51.929806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:51.929806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:47:51.929806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:47:51.929806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:47:51.929806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:47:51.930905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:51.930905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:51.930905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:47:51.930905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:51.930905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:47:51.930905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:47:51.930905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:51.930905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:51.930905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:51.930905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:51.930905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:51.930905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:51.930905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:51.930905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:51.930905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:51.930905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:51.930905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:51.930905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:51.932082 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:51.932082 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:51.932082 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:47:51.932082 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:47:51.932082 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:47:51.932082 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:47:51.932082 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:47:51.932082 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:47:51.932082 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 04:47:51.932082 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:51.935675 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 538/1091] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Jul 26 04:47:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:47:51.937313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d906b-75c0-46e6-a75a-07c27f48d7b4 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:51.969779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:52.056169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d906b-75c0-46e6-a75a-07c27f48d7b4 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:52.091403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a53ca-fee4-4266-bc6b-f4cdb76064f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:52.092065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a53ca-fee4-4266-bc6b-f4cdb76064f8 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:52.092346 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a53ca-fee4-4266-bc6b-f4cdb76064f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:52.092616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a53ca-fee4-4266-bc6b-f4cdb76064f8 None admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:52.093409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a53ca-fee4-4266-bc6b-f4cdb76064f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:52.093712 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a53ca-fee4-4266-bc6b-f4cdb76064f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:52.095150 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a53ca-fee4-4266-bc6b-f4cdb76064f8 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:52.112676 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 554/1092] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Jul 26 04:47:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:52.118638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8256d8-0cac-4d58-9682-12f338717d68 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:52.220764 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:52.246311 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8256d8-0cac-4d58-9682-12f338717d68 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:52.282276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463bffdb-0ae8-4ca4-9ebe-a7a2daf145f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:52.282905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463bffdb-0ae8-4ca4-9ebe-a7a2daf145f0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:52.283182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463bffdb-0ae8-4ca4-9ebe-a7a2daf145f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:52.283452 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463bffdb-0ae8-4ca4-9ebe-a7a2daf145f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:52.357744 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463bffdb-0ae8-4ca4-9ebe-a7a2daf145f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:52.359214 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463bffdb-0ae8-4ca4-9ebe-a7a2daf145f0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:52.368887 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9878c5-8b4e-4e07-8900-6ddcee7bccda None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:52.369189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9878c5-8b4e-4e07-8900-6ddcee7bccda None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:52.369466 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9878c5-8b4e-4e07-8900-6ddcee7bccda None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:52.370860 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 555/1093] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:47:52.376045 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27223a4-2528-4f6d-97a2-715bf68ca122 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:52.376324 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27223a4-2528-4f6d-97a2-715bf68ca122 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:52.376601 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27223a4-2528-4f6d-97a2-715bf68ca122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:52.394705 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-b27223a4-2528-4f6d-97a2-715bf68ca122 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:52.395943 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-b27223a4-2528-4f6d-97a2-715bf68ca122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:52.441754 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:52.461912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 539/1094] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:47:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 04:47:52.468038 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463bffdb-0ae8-4ca4-9ebe-a7a2daf145f0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:52.468322 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 556/1095] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 04:47:52.492138 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27223a4-2528-4f6d-97a2-715bf68ca122 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:52.506599 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:52.579290 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463bffdb-0ae8-4ca4-9ebe-a7a2daf145f0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:52.604109 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27223a4-2528-4f6d-97a2-715bf68ca122 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:52.615111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e115244-9dc7-4e11-bd48-0c349ce3405c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:52.615817 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e115244-9dc7-4e11-bd48-0c349ce3405c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:52.616100 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e115244-9dc7-4e11-bd48-0c349ce3405c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:52.616375 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e115244-9dc7-4e11-bd48-0c349ce3405c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:52.627744 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e115244-9dc7-4e11-bd48-0c349ce3405c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:52.629296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e115244-9dc7-4e11-bd48-0c349ce3405c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:52.638943 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82715b74-80ac-4928-8162-96e31f714b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:52.639658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82715b74-80ac-4928-8162-96e31f714b44 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:52.639925 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82715b74-80ac-4928-8162-96e31f714b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:52.640190 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82715b74-80ac-4928-8162-96e31f714b44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:52.644566 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-4e115244-9dc7-4e11-bd48-0c349ce3405c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:52.644566 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:52.644566 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:52.644566 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:52.644566 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:52.644566 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:52.644566 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:52.644566 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:52.644566 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:52.644566 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:52.644566 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:52.644566 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:52.644566 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:47:52.644566 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:47:52.644566 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:47:52.645837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:47:52.645837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:52.645837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:52.645837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:52.645837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:52.645837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:52.645837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:52.645837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:52.645837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:52.645837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:52.645837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:52.645837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:52.645837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:52.645837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:52.645837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:52.645837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:52.645837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:52.647302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:47:52.647302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:47:52.647302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:47:52.647302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:52.647302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 540/1096] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:47:52.651444 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e115244-9dc7-4e11-bd48-0c349ce3405c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:52.655809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82715b74-80ac-4928-8162-96e31f714b44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:47:52.660671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-82715b74-80ac-4928-8162-96e31f714b44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:47:52.660671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:52.660671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:52.660671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:52.660671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:52.660671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:52.660671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:52.660671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:52.660671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:52.660671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:52.660671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:52.660671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:52.660671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:47:52.660671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:47:52.660671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:47:52.660671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:47:52.661609 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:47:52.661609 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:47:52.661609 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:47:52.661609 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:47:52.661609 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:52.661609 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:52.661609 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:52.661609 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:52.661609 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:52.661609 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:52.661609 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:52.661609 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:52.661609 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:52.661609 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:52.661609 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:52.661609 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:52.667544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:52.667544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:52.667544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:52.667544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:52.667544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:47:52.667544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:47:52.667544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:47:52.667544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:47:52.667544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:47:52.667544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:47:52.667544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:47:52.667544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:47:52.667544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 557/1097] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:47:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:52.668291 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82715b74-80ac-4928-8162-96e31f714b44 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:52.683350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:52.740231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:52.753670 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e115244-9dc7-4e11-bd48-0c349ce3405c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:52.770783 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82715b74-80ac-4928-8162-96e31f714b44 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:52.787285 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ee8e8-6836-46d5-9582-fce3dbde23a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:52.787949 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ee8e8-6836-46d5-9582-fce3dbde23a8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:52.788228 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ee8e8-6836-46d5-9582-fce3dbde23a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:52.788501 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ee8e8-6836-46d5-9582-fce3dbde23a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:52.789285 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ee8e8-6836-46d5-9582-fce3dbde23a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:52.789583 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ee8e8-6836-46d5-9582-fce3dbde23a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:52.790951 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ee8e8-6836-46d5-9582-fce3dbde23a8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:52.802538 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 541/1098] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:52.806610 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b1d8b-a370-4e25-a769-66e9c5ccd663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:52.807241 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b1d8b-a370-4e25-a769-66e9c5ccd663 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:52.807523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b1d8b-a370-4e25-a769-66e9c5ccd663 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:52.807798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b1d8b-a370-4e25-a769-66e9c5ccd663 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:52.808344 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ee8e8-6836-46d5-9582-fce3dbde23a8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:52.808543 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b1d8b-a370-4e25-a769-66e9c5ccd663 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:52.808847 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b1d8b-a370-4e25-a769-66e9c5ccd663 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:52.810239 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b1d8b-a370-4e25-a769-66e9c5ccd663 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:52.820784 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 558/1099] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:47:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:52.826430 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b1d8b-a370-4e25-a769-66e9c5ccd663 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:52.916396 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:52.925871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ee8e8-6836-46d5-9582-fce3dbde23a8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:52.934020 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b1d8b-a370-4e25-a769-66e9c5ccd663 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:52.960239 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f50eba-301d-45b0-a3fc-05978cb6b303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:52.961039 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f50eba-301d-45b0-a3fc-05978cb6b303 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:52.961324 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f50eba-301d-45b0-a3fc-05978cb6b303 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:52.961595 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f50eba-301d-45b0-a3fc-05978cb6b303 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:52.969645 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f50eba-301d-45b0-a3fc-05978cb6b303 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:52.970737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043e2e7-2fa7-400d-8893-ac2a9946c71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:52.971393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f50eba-301d-45b0-a3fc-05978cb6b303 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:52.971495 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043e2e7-2fa7-400d-8893-ac2a9946c71a None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:52.971656 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043e2e7-2fa7-400d-8893-ac2a9946c71a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:52.971929 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043e2e7-2fa7-400d-8893-ac2a9946c71a None admin] PATH_INFO: `/v3/users/7a2fbade789949fc826de29c4f997f41` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:52.973200 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043e2e7-2fa7-400d-8893-ac2a9946c71a None admin] RBAC: Authorizing `identity:get_user(user_id=7a2fbade789949fc826de29c4f997f41)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:52.974720 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043e2e7-2fa7-400d-8893-ac2a9946c71a None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:52.976717 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 559/1100] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:47:52 2022] GET /identity/v3/users/7a2fbade789949fc826de29c4f997f41 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:52.982562 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043e2e7-2fa7-400d-8893-ac2a9946c71a None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:52.985406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-47f50eba-301d-45b0-a3fc-05978cb6b303 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:52.985406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:47:52.985406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:47:52.985406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:47:52.985406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:47:52.985406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:47:52.985406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:47:52.985406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:47:52.985406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:47:52.985406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:47:52.985406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:47:52.985406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:47:52.985406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:47:52.985406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:47:52.985406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:47:52.986466 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:47:52.986466 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:47:52.986466 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:47:52.986466 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:47:52.986466 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:47:52.986466 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:47:52.986466 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:47:52.986466 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:47:52.986466 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:47:52.986466 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:47:52.986466 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:47:52.986466 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:47:52.986466 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:47:52.986466 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:47:52.986466 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:47:52.986466 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:47:52.986466 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:47:52.987477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:47:52.987477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:47:52.987477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:47:52.987477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:47:52.987477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:47:52.987477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:47:52.987477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:47:52.987477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:47:52.987969 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:52.988040 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 542/1101] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:47:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:47:52.993071 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f50eba-301d-45b0-a3fc-05978cb6b303 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:53.087175 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043e2e7-2fa7-400d-8893-ac2a9946c71a None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:53.097656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f50eba-301d-45b0-a3fc-05978cb6b303 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:53.120708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18244cc4-0116-4fdb-98d9-ada4a804d391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:53.121329 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18244cc4-0116-4fdb-98d9-ada4a804d391 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:53.121605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18244cc4-0116-4fdb-98d9-ada4a804d391 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:53.121877 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18244cc4-0116-4fdb-98d9-ada4a804d391 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:53.131286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c689c-d5c4-43b6-a06f-a657ba9f0ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:53.131947 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c689c-d5c4-43b6-a06f-a657ba9f0ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:53.132234 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c689c-d5c4-43b6-a06f-a657ba9f0ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:53.132510 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c689c-d5c4-43b6-a06f-a657ba9f0ae5 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:53.133296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c689c-d5c4-43b6-a06f-a657ba9f0ae5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:53.133606 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c689c-d5c4-43b6-a06f-a657ba9f0ae5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:53.135064 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c689c-d5c4-43b6-a06f-a657ba9f0ae5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:53.146515 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 543/1102] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:47:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:53.152817 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c689c-d5c4-43b6-a06f-a657ba9f0ae5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:53.168541 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:53.190210 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18244cc4-0116-4fdb-98d9-ada4a804d391 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:53.191738 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18244cc4-0116-4fdb-98d9-ada4a804d391 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:53.193698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:53.254984 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c689c-d5c4-43b6-a06f-a657ba9f0ae5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:53.292784 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 560/1103] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:47:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 04:47:53.298293 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82911626-7034-4c22-a682-b5006865541d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:53.298638 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18244cc4-0116-4fdb-98d9-ada4a804d391 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:53.298921 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82911626-7034-4c22-a682-b5006865541d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:53.299197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82911626-7034-4c22-a682-b5006865541d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:53.299463 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82911626-7034-4c22-a682-b5006865541d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:53.300283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82911626-7034-4c22-a682-b5006865541d None admin] RBAC: Adding query filter params (role.id=29516d985a494f4b88d90e1e4f525c24, scope.project.id=d583e0be8ef74766bffd6c3c07af18f1, user.id=2faeed9002ee4154bc980c146cd06bcc) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:53.300586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82911626-7034-4c22-a682-b5006865541d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:53.302144 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82911626-7034-4c22-a682-b5006865541d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:53.311629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 544/1104] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:47:53 2022] GET /identity/v3/role_assignments?user.id=2faeed9002ee4154bc980c146cd06bcc&scope.project.id=d583e0be8ef74766bffd6c3c07af18f1&role.id=29516d985a494f4b88d90e1e4f525c24 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:53.413615 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18244cc4-0116-4fdb-98d9-ada4a804d391 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:53.415432 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:53.455448 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816187f7-4a42-403e-9293-7f6b75e926f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:53.456141 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816187f7-4a42-403e-9293-7f6b75e926f6 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:53.456425 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816187f7-4a42-403e-9293-7f6b75e926f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:53.456693 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816187f7-4a42-403e-9293-7f6b75e926f6 None admin] PATH_INFO: `/v3/projects/2f7110dfeaa74ec1aecff6d2cad32543` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:53.458012 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816187f7-4a42-403e-9293-7f6b75e926f6 None admin] RBAC: Authorizing `identity:get_project(project_id=2f7110dfeaa74ec1aecff6d2cad32543)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:53.459480 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816187f7-4a42-403e-9293-7f6b75e926f6 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:53.461251 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 561/1105] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:47:53 2022] GET /identity/v3/projects/2f7110dfeaa74ec1aecff6d2cad32543 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:47:53.468170 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82911626-7034-4c22-a682-b5006865541d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:53.549843 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:53.573884 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82911626-7034-4c22-a682-b5006865541d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:53.608547 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ff2af-d61c-4ec6-b1e5-a868f995357f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:53.609156 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ff2af-d61c-4ec6-b1e5-a868f995357f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:53.609473 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ff2af-d61c-4ec6-b1e5-a868f995357f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:53.609716 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ff2af-d61c-4ec6-b1e5-a868f995357f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:53.610539 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ff2af-d61c-4ec6-b1e5-a868f995357f None admin] RBAC: Adding query filter params (role.id=72bf8bd23d2c4b76ad32b7b35d6db412, scope.project.id=2f7110dfeaa74ec1aecff6d2cad32543, user.id=7a2fbade789949fc826de29c4f997f41) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:53.610845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ff2af-d61c-4ec6-b1e5-a868f995357f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:53.612330 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ff2af-d61c-4ec6-b1e5-a868f995357f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:53.619157 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 545/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:47:53 2022] GET /identity/v3/role_assignments?user.id=7a2fbade789949fc826de29c4f997f41&scope.project.id=2f7110dfeaa74ec1aecff6d2cad32543&role.id=72bf8bd23d2c4b76ad32b7b35d6db412 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:55.630434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4df699-6f37-4996-8264-d7f1dc066405 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:55.631053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4df699-6f37-4996-8264-d7f1dc066405 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:55.631053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4df699-6f37-4996-8264-d7f1dc066405 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:55.632415 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 562/1107] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:55.638156 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5e32e-e8d5-4930-8be8-86bbf53cea07 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:55.638630 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5e32e-e8d5-4930-8be8-86bbf53cea07 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:55.638733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5e32e-e8d5-4930-8be8-86bbf53cea07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:55.660912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2ff5e32e-e8d5-4930-8be8-86bbf53cea07 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:55.662048 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2ff5e32e-e8d5-4930-8be8-86bbf53cea07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:55.735069 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 546/1108] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:55.759504 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4df699-6f37-4996-8264-d7f1dc066405 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:55.872285 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4df699-6f37-4996-8264-d7f1dc066405 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:55.907148 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce29308-d3b7-465f-9357-68c0797ac8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:55.907823 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce29308-d3b7-465f-9357-68c0797ac8fb None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:55.908106 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce29308-d3b7-465f-9357-68c0797ac8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:55.908380 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce29308-d3b7-465f-9357-68c0797ac8fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:55.909588 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce29308-d3b7-465f-9357-68c0797ac8fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:55.911101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce29308-d3b7-465f-9357-68c0797ac8fb None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:55.912700 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 563/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:47:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:55.919028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5e32e-e8d5-4930-8be8-86bbf53cea07 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:55.987825 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:56.021242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5e32e-e8d5-4930-8be8-86bbf53cea07 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:56.060725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f69b10-4b27-4a98-9b4a-64084ea42b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:56.061270 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f69b10-4b27-4a98-9b4a-64084ea42b7a None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:56.061549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f69b10-4b27-4a98-9b4a-64084ea42b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:56.061816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f69b10-4b27-4a98-9b4a-64084ea42b7a None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:56.062500 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f69b10-4b27-4a98-9b4a-64084ea42b7a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:56.063976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f69b10-4b27-4a98-9b4a-64084ea42b7a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:56.087904 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 547/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jul 26 04:47:55 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:47:58.144055 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d601f-0acf-4bec-810d-27615fa9a6db None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:58.144654 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d601f-0acf-4bec-810d-27615fa9a6db None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:58.144654 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d601f-0acf-4bec-810d-27615fa9a6db None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:58.145963 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 564/1111] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:47:58.151465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2254f14-48e4-46e6-b66c-57484c3a795e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:58.152004 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2254f14-48e4-46e6-b66c-57484c3a795e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:58.152004 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2254f14-48e4-46e6-b66c-57484c3a795e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:58.171285 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c2254f14-48e4-46e6-b66c-57484c3a795e None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:47:58.172493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c2254f14-48e4-46e6-b66c-57484c3a795e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:47:58.242898 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 548/1112] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:47:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:47:58.266272 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d601f-0acf-4bec-810d-27615fa9a6db None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:58.312344 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:58.378476 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d601f-0acf-4bec-810d-27615fa9a6db None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:58.414925 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeadd68-b37b-4196-a006-339ca31f40c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:58.415506 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeadd68-b37b-4196-a006-339ca31f40c3 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:58.415832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeadd68-b37b-4196-a006-339ca31f40c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:58.416106 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeadd68-b37b-4196-a006-339ca31f40c3 None admin] PATH_INFO: `/v3/roles/9223a9bb40484faca3a7bcfeb41a2215` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:58.417599 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeadd68-b37b-4196-a006-339ca31f40c3 None admin] RBAC: Authorizing `identity:get_role(role_id=9223a9bb40484faca3a7bcfeb41a2215)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:58.418969 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeadd68-b37b-4196-a006-339ca31f40c3 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:58.420422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 565/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:47:58 2022] GET /identity/v3/roles/9223a9bb40484faca3a7bcfeb41a2215 => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:58.430003 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2254f14-48e4-46e6-b66c-57484c3a795e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:58.488633 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:58.535290 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2254f14-48e4-46e6-b66c-57484c3a795e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:58.570438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a04dcf-234c-47dc-bfa1-8edf836a6109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:58.571087 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a04dcf-234c-47dc-bfa1-8edf836a6109 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:58.571367 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a04dcf-234c-47dc-bfa1-8edf836a6109 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:58.571682 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a04dcf-234c-47dc-bfa1-8edf836a6109 None admin] PATH_INFO: `/v3/users/a739a18cbe854da2acb7a69a3a4c2d8a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:58.586153 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a04dcf-234c-47dc-bfa1-8edf836a6109 None admin] RBAC: Authorizing `identity:get_user(user_id=a739a18cbe854da2acb7a69a3a4c2d8a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:58.587647 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a04dcf-234c-47dc-bfa1-8edf836a6109 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:58.589715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 549/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:47:58 2022] GET /identity/v3/users/a739a18cbe854da2acb7a69a3a4c2d8a => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:58.595857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeadd68-b37b-4196-a006-339ca31f40c3 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:58.700354 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeadd68-b37b-4196-a006-339ca31f40c3 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:58.707655 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:58.741713 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb33a2-8b07-47e6-94b7-80104f59bca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:58.742341 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb33a2-8b07-47e6-94b7-80104f59bca3 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:58.742629 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb33a2-8b07-47e6-94b7-80104f59bca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:58.742904 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb33a2-8b07-47e6-94b7-80104f59bca3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:58.811951 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beb33a2-8b07-47e6-94b7-80104f59bca3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:58.813487 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beb33a2-8b07-47e6-94b7-80104f59bca3 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:58.916852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 566/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:47:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 04:47:58.923523 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a04dcf-234c-47dc-bfa1-8edf836a6109 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:59.033165 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a04dcf-234c-47dc-bfa1-8edf836a6109 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:59.047674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:59.069185 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17adab-7b37-44a0-988a-517fa9de344e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:59.069840 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17adab-7b37-44a0-988a-517fa9de344e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:59.070125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17adab-7b37-44a0-988a-517fa9de344e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:59.070402 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17adab-7b37-44a0-988a-517fa9de344e None admin] PATH_INFO: `/v3/projects/2f7110dfeaa74ec1aecff6d2cad32543` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:59.071614 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17adab-7b37-44a0-988a-517fa9de344e None admin] RBAC: Authorizing `identity:get_project(project_id=2f7110dfeaa74ec1aecff6d2cad32543)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:59.073071 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17adab-7b37-44a0-988a-517fa9de344e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:59.074802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 550/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:47:58 2022] GET /identity/v3/projects/2f7110dfeaa74ec1aecff6d2cad32543 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:47:59.081277 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb33a2-8b07-47e6-94b7-80104f59bca3 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:47:59.090124 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:47:59.185195 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb33a2-8b07-47e6-94b7-80104f59bca3 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:47:59.220561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f88f71-4a9f-43b5-b6f1-8c8dbcc1e660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:47:59.221183 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f88f71-4a9f-43b5-b6f1-8c8dbcc1e660 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:47:59.221483 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f88f71-4a9f-43b5-b6f1-8c8dbcc1e660 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:47:59.221753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f88f71-4a9f-43b5-b6f1-8c8dbcc1e660 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:47:59.222558 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f88f71-4a9f-43b5-b6f1-8c8dbcc1e660 None admin] RBAC: Adding query filter params (role.id=9223a9bb40484faca3a7bcfeb41a2215, scope.project.id=2f7110dfeaa74ec1aecff6d2cad32543, user.id=a739a18cbe854da2acb7a69a3a4c2d8a) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:47:59.222870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f88f71-4a9f-43b5-b6f1-8c8dbcc1e660 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:47:59.224357 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f88f71-4a9f-43b5-b6f1-8c8dbcc1e660 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:47:59.231152 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 567/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:47:59 2022] GET /identity/v3/role_assignments?user.id=a739a18cbe854da2acb7a69a3a4c2d8a&scope.project.id=2f7110dfeaa74ec1aecff6d2cad32543&role.id=9223a9bb40484faca3a7bcfeb41a2215 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:01.178607 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50d0f6-51f8-473e-8d12-4f212699712f None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:01.179253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50d0f6-51f8-473e-8d12-4f212699712f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:01.179253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50d0f6-51f8-473e-8d12-4f212699712f None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:01.180666 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 551/1118] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:48:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:48:01.186308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0eb22-12aa-47c8-b7b2-ba7d636b1137 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:01.186746 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0eb22-12aa-47c8-b7b2-ba7d636b1137 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:01.186850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0eb22-12aa-47c8-b7b2-ba7d636b1137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:01.206885 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-19c0eb22-12aa-47c8-b7b2-ba7d636b1137 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:48:01.208104 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-19c0eb22-12aa-47c8-b7b2-ba7d636b1137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:48:01.275113 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:01.280567 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 568/1119] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:48:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:48:01.302789 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50d0f6-51f8-473e-8d12-4f212699712f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:01.431194 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50d0f6-51f8-473e-8d12-4f212699712f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:01.467719 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25188c63-2c70-4fad-afc0-68a41a223d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:01.468394 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25188c63-2c70-4fad-afc0-68a41a223d7d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:01.468685 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25188c63-2c70-4fad-afc0-68a41a223d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:01.469025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25188c63-2c70-4fad-afc0-68a41a223d7d None admin] PATH_INFO: `/v3/roles/9223a9bb40484faca3a7bcfeb41a2215` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:01.470541 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25188c63-2c70-4fad-afc0-68a41a223d7d None admin] RBAC: Authorizing `identity:get_role(role_id=9223a9bb40484faca3a7bcfeb41a2215)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:01.472048 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25188c63-2c70-4fad-afc0-68a41a223d7d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:01.473509 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 552/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:48:01 2022] GET /identity/v3/roles/9223a9bb40484faca3a7bcfeb41a2215 => generated 223 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:01.480071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0eb22-12aa-47c8-b7b2-ba7d636b1137 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:01.583206 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0eb22-12aa-47c8-b7b2-ba7d636b1137 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:01.619710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569507eb-80c4-4031-8d84-e7c4f39084ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:01.620358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569507eb-80c4-4031-8d84-e7c4f39084ae None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:01.620643 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569507eb-80c4-4031-8d84-e7c4f39084ae None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:01.620919 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569507eb-80c4-4031-8d84-e7c4f39084ae None admin] PATH_INFO: `/v3/users/a739a18cbe854da2acb7a69a3a4c2d8a` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:01.622304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569507eb-80c4-4031-8d84-e7c4f39084ae None admin] RBAC: Authorizing `identity:get_user(user_id=a739a18cbe854da2acb7a69a3a4c2d8a)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:01.623605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569507eb-80c4-4031-8d84-e7c4f39084ae None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:01.625274 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 569/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:48:01 2022] GET /identity/v3/users/a739a18cbe854da2acb7a69a3a4c2d8a => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:01.632076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25188c63-2c70-4fad-afc0-68a41a223d7d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:01.696420 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:01.751314 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25188c63-2c70-4fad-afc0-68a41a223d7d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:01.787054 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b39da07-24a7-4716-b8c9-3ce566b0d6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:01.787723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39da07-24a7-4716-b8c9-3ce566b0d6fa None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:01.788005 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39da07-24a7-4716-b8c9-3ce566b0d6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:01.788272 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39da07-24a7-4716-b8c9-3ce566b0d6fa None admin] PATH_INFO: `/v3/projects/2f7110dfeaa74ec1aecff6d2cad32543` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:01.789588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b39da07-24a7-4716-b8c9-3ce566b0d6fa None admin] RBAC: Authorizing `identity:get_project(project_id=2f7110dfeaa74ec1aecff6d2cad32543)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:01.791028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b39da07-24a7-4716-b8c9-3ce566b0d6fa None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:01.792762 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 553/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:48:01 2022] GET /identity/v3/projects/2f7110dfeaa74ec1aecff6d2cad32543 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:01.798967 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569507eb-80c4-4031-8d84-e7c4f39084ae None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:01.871938 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:01.908592 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569507eb-80c4-4031-8d84-e7c4f39084ae None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:01.944851 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616f366-4f85-4c12-9192-699de8064c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:01.945520 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616f366-4f85-4c12-9192-699de8064c18 None admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:01.945801 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616f366-4f85-4c12-9192-699de8064c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:01.946066 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616f366-4f85-4c12-9192-699de8064c18 None admin] PATH_INFO: `/v3/projects/2f7110dfeaa74ec1aecff6d2cad32543/users/a739a18cbe854da2acb7a69a3a4c2d8a/roles/9223a9bb40484faca3a7bcfeb41a2215` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:01.948508 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616f366-4f85-4c12-9192-699de8064c18 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f7110dfeaa74ec1aecff6d2cad32543, user_id=a739a18cbe854da2acb7a69a3a4c2d8a, role_id=9223a9bb40484faca3a7bcfeb41a2215)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:01.949908 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616f366-4f85-4c12-9192-699de8064c18 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:01.964614 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 570/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jul 26 04:48:01 2022] PUT /identity/v3/projects/2f7110dfeaa74ec1aecff6d2cad32543/users/a739a18cbe854da2acb7a69a3a4c2d8a/roles/9223a9bb40484faca3a7bcfeb41a2215 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:48:03.929307 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209646a1-7582-42fc-93cd-3525a9e11c0c None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:03.929864 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209646a1-7582-42fc-93cd-3525a9e11c0c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:03.929864 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209646a1-7582-42fc-93cd-3525a9e11c0c None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:03.931230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 554/1124] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:48:03.936643 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ad694-a82a-4762-9686-530f50e28297 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:03.937093 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ad694-a82a-4762-9686-530f50e28297 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:03.937187 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ad694-a82a-4762-9686-530f50e28297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:03.956609 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-4d8ad694-a82a-4762-9686-530f50e28297 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:48:03.957949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-4d8ad694-a82a-4762-9686-530f50e28297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:48:04.031426 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 571/1125] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:48:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:48:04.054916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209646a1-7582-42fc-93cd-3525a9e11c0c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:04.161563 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209646a1-7582-42fc-93cd-3525a9e11c0c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:04.179826 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:04.198463 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ccf66-92c2-4ddb-9340-340d0b91196b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:04.199109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ccf66-92c2-4ddb-9340-340d0b91196b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:04.199392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ccf66-92c2-4ddb-9340-340d0b91196b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:04.199699 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ccf66-92c2-4ddb-9340-340d0b91196b None admin] PATH_INFO: `/v3/roles/9223a9bb40484faca3a7bcfeb41a2215` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:04.201178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674ccf66-92c2-4ddb-9340-340d0b91196b None admin] RBAC: Authorizing `identity:get_role(role_id=9223a9bb40484faca3a7bcfeb41a2215)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:04.202558 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674ccf66-92c2-4ddb-9340-340d0b91196b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:04.203985 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 555/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:48:04 2022] GET /identity/v3/roles/9223a9bb40484faca3a7bcfeb41a2215 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:04.210186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ad694-a82a-4762-9686-530f50e28297 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:04.269727 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:04.311764 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ad694-a82a-4762-9686-530f50e28297 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:04.347574 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e975d-7ddd-43e1-ac39-c0da74d8b243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:04.348243 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e975d-7ddd-43e1-ac39-c0da74d8b243 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:04.348525 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e975d-7ddd-43e1-ac39-c0da74d8b243 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:04.348796 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e975d-7ddd-43e1-ac39-c0da74d8b243 None admin] PATH_INFO: `/v3/users/a739a18cbe854da2acb7a69a3a4c2d8a` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:04.349985 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e975d-7ddd-43e1-ac39-c0da74d8b243 None admin] RBAC: Authorizing `identity:get_user(user_id=a739a18cbe854da2acb7a69a3a4c2d8a)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:04.351516 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e975d-7ddd-43e1-ac39-c0da74d8b243 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:04.353196 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 572/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:48:04 2022] GET /identity/v3/users/a739a18cbe854da2acb7a69a3a4c2d8a => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:04.359161 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ccf66-92c2-4ddb-9340-340d0b91196b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:04.465395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ccf66-92c2-4ddb-9340-340d0b91196b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:04.500461 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01079cc-ed0e-45ee-9733-183b67820d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:04.501119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01079cc-ed0e-45ee-9733-183b67820d6a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:04.501400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01079cc-ed0e-45ee-9733-183b67820d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:04.501683 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01079cc-ed0e-45ee-9733-183b67820d6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:04.558458 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:04.573136 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01079cc-ed0e-45ee-9733-183b67820d6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:04.574600 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01079cc-ed0e-45ee-9733-183b67820d6a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:04.675172 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 556/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:48:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 04:48:04.681420 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e975d-7ddd-43e1-ac39-c0da74d8b243 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:04.784625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e975d-7ddd-43e1-ac39-c0da74d8b243 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:04.809825 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:04.821050 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cded1b-d01c-4032-b830-761371666391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:04.821827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cded1b-d01c-4032-b830-761371666391 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:04.821972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cded1b-d01c-4032-b830-761371666391 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:04.822254 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cded1b-d01c-4032-b830-761371666391 None admin] PATH_INFO: `/v3/projects/2f7110dfeaa74ec1aecff6d2cad32543` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:04.823475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cded1b-d01c-4032-b830-761371666391 None admin] RBAC: Authorizing `identity:get_project(project_id=2f7110dfeaa74ec1aecff6d2cad32543)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:04.824969 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cded1b-d01c-4032-b830-761371666391 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:04.826664 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 573/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:48:04 2022] GET /identity/v3/projects/2f7110dfeaa74ec1aecff6d2cad32543 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:04.833128 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01079cc-ed0e-45ee-9733-183b67820d6a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:04.920976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:04.938944 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01079cc-ed0e-45ee-9733-183b67820d6a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:04.977638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360b8dd-1ea3-49ba-bb50-63e1ff840a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:04.978239 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360b8dd-1ea3-49ba-bb50-63e1ff840a3b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:04.978524 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360b8dd-1ea3-49ba-bb50-63e1ff840a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:04.978797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360b8dd-1ea3-49ba-bb50-63e1ff840a3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:04.979610 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f360b8dd-1ea3-49ba-bb50-63e1ff840a3b None admin] RBAC: Adding query filter params (role.id=9223a9bb40484faca3a7bcfeb41a2215, scope.project.id=2f7110dfeaa74ec1aecff6d2cad32543, user.id=a739a18cbe854da2acb7a69a3a4c2d8a) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:48:04.979923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f360b8dd-1ea3-49ba-bb50-63e1ff840a3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:04.981448 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f360b8dd-1ea3-49ba-bb50-63e1ff840a3b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:04.987734 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 557/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:48:04 2022] GET /identity/v3/role_assignments?user.id=a739a18cbe854da2acb7a69a3a4c2d8a&scope.project.id=2f7110dfeaa74ec1aecff6d2cad32543&role.id=9223a9bb40484faca3a7bcfeb41a2215 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:06.991336 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee7d7d-7085-424d-882c-3b4595a9c1b3 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:06.991935 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee7d7d-7085-424d-882c-3b4595a9c1b3 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:06.991935 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee7d7d-7085-424d-882c-3b4595a9c1b3 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:06.993377 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 574/1131] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:48:06.998901 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365148d8-dc99-4d0c-99a1-fa9ac490fd88 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:06.999348 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365148d8-dc99-4d0c-99a1-fa9ac490fd88 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:06.999439 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365148d8-dc99-4d0c-99a1-fa9ac490fd88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:07.018926 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-365148d8-dc99-4d0c-99a1-fa9ac490fd88 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:48:07.020298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-365148d8-dc99-4d0c-99a1-fa9ac490fd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:48:07.095956 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 558/1132] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:48:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:48:07.118703 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee7d7d-7085-424d-882c-3b4595a9c1b3 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:07.227572 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee7d7d-7085-424d-882c-3b4595a9c1b3 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:07.266032 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a70fb-e7c3-4c36-9431-531c8ba473ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:07.266599 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a70fb-e7c3-4c36-9431-531c8ba473ad None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:07.266880 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a70fb-e7c3-4c36-9431-531c8ba473ad None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:07.267152 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a70fb-e7c3-4c36-9431-531c8ba473ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:07.268389 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737a70fb-e7c3-4c36-9431-531c8ba473ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:07.269853 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737a70fb-e7c3-4c36-9431-531c8ba473ad None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:07.271493 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 575/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:48:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:07.278048 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365148d8-dc99-4d0c-99a1-fa9ac490fd88 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:07.364317 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:07.386400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365148d8-dc99-4d0c-99a1-fa9ac490fd88 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:07.424714 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4c8f6-a745-4615-85a9-0f46b369d7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:07.425353 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4c8f6-a745-4615-85a9-0f46b369d7e2 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:07.425638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4c8f6-a745-4615-85a9-0f46b369d7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:07.425916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4c8f6-a745-4615-85a9-0f46b369d7e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:07.426623 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4c8f6-a745-4615-85a9-0f46b369d7e2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:07.428111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4c8f6-a745-4615-85a9-0f46b369d7e2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:07.463823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 559/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jul 26 04:48:07 2022] POST /identity/v3/projects => generated 311 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:48:09.491784 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7674bb-d328-4e7e-8b97-b067db76c2b8 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:09.492339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7674bb-d328-4e7e-8b97-b067db76c2b8 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:09.492339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7674bb-d328-4e7e-8b97-b067db76c2b8 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:09.493700 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 576/1135] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:48:09.499023 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4407ea-08ed-442c-b924-e20f17a016ed None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:09.499469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4407ea-08ed-442c-b924-e20f17a016ed None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:09.499589 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4407ea-08ed-442c-b924-e20f17a016ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:09.518211 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4d4407ea-08ed-442c-b924-e20f17a016ed None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:48:09.519391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4d4407ea-08ed-442c-b924-e20f17a016ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:48:09.590000 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 560/1136] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:48:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:48:09.615282 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7674bb-d328-4e7e-8b97-b067db76c2b8 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:09.682582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:09.722546 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7674bb-d328-4e7e-8b97-b067db76c2b8 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:09.757906 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71fe635-1456-4a1a-98b4-4987cac345b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:09.758547 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71fe635-1456-4a1a-98b4-4987cac345b0 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:09.758830 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71fe635-1456-4a1a-98b4-4987cac345b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:09.759100 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71fe635-1456-4a1a-98b4-4987cac345b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:09.760324 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71fe635-1456-4a1a-98b4-4987cac345b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:09.761787 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71fe635-1456-4a1a-98b4-4987cac345b0 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:09.763402 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 577/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:48:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:09.769778 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4407ea-08ed-442c-b924-e20f17a016ed None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:09.835008 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:09.876421 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4407ea-08ed-442c-b924-e20f17a016ed None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:09.911682 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eca600-693d-4010-ab93-fa86a871b96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:09.912314 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eca600-693d-4010-ab93-fa86a871b96a None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:09.912595 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eca600-693d-4010-ab93-fa86a871b96a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:09.912871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eca600-693d-4010-ab93-fa86a871b96a None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:09.913616 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eca600-693d-4010-ab93-fa86a871b96a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:09.915065 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eca600-693d-4010-ab93-fa86a871b96a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:09.938340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 561/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jul 26 04:48:09 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:48:11.996305 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a6f37-c2a5-4a56-9ce7-fc3d83a71316 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:11.996871 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a6f37-c2a5-4a56-9ce7-fc3d83a71316 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:11.996871 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a6f37-c2a5-4a56-9ce7-fc3d83a71316 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:11.998232 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 578/1139] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:48:12.003640 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdca633-fa59-480d-99df-2e48d775f75d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:12.004100 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdca633-fa59-480d-99df-2e48d775f75d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:12.004200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdca633-fa59-480d-99df-2e48d775f75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:12.023352 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2fdca633-fa59-480d-99df-2e48d775f75d None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:48:12.024611 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2fdca633-fa59-480d-99df-2e48d775f75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:48:12.096181 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 562/1140] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:48:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:48:12.120040 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a6f37-c2a5-4a56-9ce7-fc3d83a71316 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:12.230832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a6f37-c2a5-4a56-9ce7-fc3d83a71316 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:12.260818 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:12.267835 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117d4fd-d705-479d-b29b-fc82a4ede418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:12.268493 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117d4fd-d705-479d-b29b-fc82a4ede418 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:12.268780 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117d4fd-d705-479d-b29b-fc82a4ede418 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:12.269057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117d4fd-d705-479d-b29b-fc82a4ede418 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:12.280327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9117d4fd-d705-479d-b29b-fc82a4ede418 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:48:12.285219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-9117d4fd-d705-479d-b29b-fc82a4ede418 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:48:12.285219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:48:12.285219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:48:12.285219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:48:12.285219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:48:12.285219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:48:12.285219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:48:12.285219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:48:12.285219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:48:12.285219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:48:12.285219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:48:12.285219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:48:12.285219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:48:12.285219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:48:12.285219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:48:12.285219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:48:12.286465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:48:12.286465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:48:12.286465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:48:12.286465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:48:12.286465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:48:12.286465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:48:12.286465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:48:12.286465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:48:12.286465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:48:12.286465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:48:12.286465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:48:12.286465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:48:12.286465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:48:12.286465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:48:12.286465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:48:12.286465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:48:12.287387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:48:12.287387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:48:12.287387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:48:12.287387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:48:12.287387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:48:12.287387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:48:12.287387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:48:12.287387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:48:12.287387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:48:12.287387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:48:12.287387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:48:12.287387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:48:12.287387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 579/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:48:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:48:12.293133 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdca633-fa59-480d-99df-2e48d775f75d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:12.345695 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:12.398167 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdca633-fa59-480d-99df-2e48d775f75d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:12.434381 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c5c91-d609-48c1-a761-caefc3621df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:12.435012 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c5c91-d609-48c1-a761-caefc3621df7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:12.435300 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c5c91-d609-48c1-a761-caefc3621df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:12.435588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c5c91-d609-48c1-a761-caefc3621df7 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:12.436326 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c5c91-d609-48c1-a761-caefc3621df7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:48:12.436640 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c5c91-d609-48c1-a761-caefc3621df7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:12.438022 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c5c91-d609-48c1-a761-caefc3621df7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:12.446657 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 563/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:48:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:12.452813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117d4fd-d705-479d-b29b-fc82a4ede418 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:12.558738 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117d4fd-d705-479d-b29b-fc82a4ede418 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:12.594926 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab19d6-f494-41bf-8814-b3263835586f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:12.595593 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab19d6-f494-41bf-8814-b3263835586f None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:12.595884 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab19d6-f494-41bf-8814-b3263835586f None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:12.596157 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab19d6-f494-41bf-8814-b3263835586f None admin] PATH_INFO: `/v3/users/da54699c55b7426dbc4a7f033eb68db6` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:12.611590 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab19d6-f494-41bf-8814-b3263835586f None admin] RBAC: Authorizing `identity:get_user(user_id=da54699c55b7426dbc4a7f033eb68db6)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:12.613654 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab19d6-f494-41bf-8814-b3263835586f None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:12.615400 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 580/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:48:12 2022] GET /identity/v3/users/da54699c55b7426dbc4a7f033eb68db6 => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:12.621952 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c5c91-d609-48c1-a761-caefc3621df7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:12.730768 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c5c91-d609-48c1-a761-caefc3621df7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:12.741681 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:12.768566 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47266bde-0846-42fb-adaa-8b6284836589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:12.769202 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47266bde-0846-42fb-adaa-8b6284836589 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:12.769483 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47266bde-0846-42fb-adaa-8b6284836589 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:12.769758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47266bde-0846-42fb-adaa-8b6284836589 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:12.840458 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47266bde-0846-42fb-adaa-8b6284836589 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:12.841835 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47266bde-0846-42fb-adaa-8b6284836589 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:12.961295 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 564/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:48:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 04:48:12.967878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab19d6-f494-41bf-8814-b3263835586f None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:13.024337 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:13.076521 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab19d6-f494-41bf-8814-b3263835586f None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:13.113674 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed9d1ba-bdfe-451f-a825-6f7b917ffc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:13.114248 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9d1ba-bdfe-451f-a825-6f7b917ffc24 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:13.114529 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9d1ba-bdfe-451f-a825-6f7b917ffc24 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:13.114806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9d1ba-bdfe-451f-a825-6f7b917ffc24 None admin] PATH_INFO: `/v3/projects/1e667793dbb14da0aef46a24d0efaf44` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:13.116126 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed9d1ba-bdfe-451f-a825-6f7b917ffc24 None admin] RBAC: Authorizing `identity:get_project(project_id=1e667793dbb14da0aef46a24d0efaf44)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:13.117613 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed9d1ba-bdfe-451f-a825-6f7b917ffc24 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:13.119388 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 581/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:48:12 2022] GET /identity/v3/projects/1e667793dbb14da0aef46a24d0efaf44 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:13.126047 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47266bde-0846-42fb-adaa-8b6284836589 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:13.138162 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:13.230355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47266bde-0846-42fb-adaa-8b6284836589 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:13.266051 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706efe3-17aa-4935-b6a2-1f0f3aecaa95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:13.266676 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706efe3-17aa-4935-b6a2-1f0f3aecaa95 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:13.266961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706efe3-17aa-4935-b6a2-1f0f3aecaa95 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:13.267235 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706efe3-17aa-4935-b6a2-1f0f3aecaa95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:13.268059 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706efe3-17aa-4935-b6a2-1f0f3aecaa95 None admin] RBAC: Adding query filter params (role.id=72bf8bd23d2c4b76ad32b7b35d6db412, scope.project.id=1e667793dbb14da0aef46a24d0efaf44, user.id=da54699c55b7426dbc4a7f033eb68db6) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:48:13.268371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706efe3-17aa-4935-b6a2-1f0f3aecaa95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:13.269837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706efe3-17aa-4935-b6a2-1f0f3aecaa95 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:13.286212 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 565/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:48:13 2022] GET /identity/v3/role_assignments?user.id=da54699c55b7426dbc4a7f033eb68db6&scope.project.id=1e667793dbb14da0aef46a24d0efaf44&role.id=72bf8bd23d2c4b76ad32b7b35d6db412 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:15.247146 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89202e-3b3f-4d69-803c-cd71c34dc5e2 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:15.247740 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89202e-3b3f-4d69-803c-cd71c34dc5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:15.247740 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89202e-3b3f-4d69-803c-cd71c34dc5e2 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:15.249076 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 582/1147] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:48:15.254572 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b1368-499d-4638-929f-323b4899dacd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:15.255030 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b1368-499d-4638-929f-323b4899dacd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:15.255123 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b1368-499d-4638-929f-323b4899dacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:15.274724 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-534b1368-499d-4638-929f-323b4899dacd None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:48:15.276000 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-534b1368-499d-4638-929f-323b4899dacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:48:15.349548 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 566/1148] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:48:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:48:15.371824 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89202e-3b3f-4d69-803c-cd71c34dc5e2 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:15.476870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89202e-3b3f-4d69-803c-cd71c34dc5e2 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:15.493370 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:15.513086 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be120446-3b94-47ea-8eaf-ac2ad8151f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:15.513665 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be120446-3b94-47ea-8eaf-ac2ad8151f8a None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:15.513948 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be120446-3b94-47ea-8eaf-ac2ad8151f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:15.514225 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be120446-3b94-47ea-8eaf-ac2ad8151f8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:15.525182 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be120446-3b94-47ea-8eaf-ac2ad8151f8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:48:15.530028 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-be120446-3b94-47ea-8eaf-ac2ad8151f8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:48:15.530028 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:48:15.530028 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:48:15.530028 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:48:15.530028 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:48:15.530028 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:48:15.530028 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:48:15.530028 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:48:15.530028 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:48:15.530028 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:48:15.530028 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:48:15.530028 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:48:15.530028 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:48:15.530028 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:48:15.530028 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:48:15.530028 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:48:15.531192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:48:15.531192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:48:15.531192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:48:15.531192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:48:15.531192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:48:15.531192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:48:15.531192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:48:15.531192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:48:15.531192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:48:15.531192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:48:15.531192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:48:15.531192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:48:15.531192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:48:15.531192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:48:15.531192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:48:15.531192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:48:15.534671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:48:15.534671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:48:15.534671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:48:15.534671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:48:15.534671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:48:15.534671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:48:15.534671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:48:15.534671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:48:15.534671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:48:15.534671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:48:15.534671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:48:15.534671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:48:15.534671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 583/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:48:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:48:15.537700 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b1368-499d-4638-929f-323b4899dacd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:15.541011 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:15.641596 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b1368-499d-4638-929f-323b4899dacd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:15.684805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc4e8a-5b97-484f-873c-af30d96fbbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:15.685429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc4e8a-5b97-484f-873c-af30d96fbbe4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:15.685708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc4e8a-5b97-484f-873c-af30d96fbbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:15.685978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc4e8a-5b97-484f-873c-af30d96fbbe4 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:15.686694 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc4e8a-5b97-484f-873c-af30d96fbbe4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:48:15.686992 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc4e8a-5b97-484f-873c-af30d96fbbe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:15.688382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc4e8a-5b97-484f-873c-af30d96fbbe4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:15.696929 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 567/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:48:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:15.703112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be120446-3b94-47ea-8eaf-ac2ad8151f8a None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:15.815494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be120446-3b94-47ea-8eaf-ac2ad8151f8a None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:15.855768 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c4223-b66c-4732-96e4-55a3966f7c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:15.856428 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c4223-b66c-4732-96e4-55a3966f7c5a None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:15.856711 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c4223-b66c-4732-96e4-55a3966f7c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:15.856990 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c4223-b66c-4732-96e4-55a3966f7c5a None admin] PATH_INFO: `/v3/users/da54699c55b7426dbc4a7f033eb68db6` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:15.858290 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c4223-b66c-4732-96e4-55a3966f7c5a None admin] RBAC: Authorizing `identity:get_user(user_id=da54699c55b7426dbc4a7f033eb68db6)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:15.859798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c4223-b66c-4732-96e4-55a3966f7c5a None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:15.861454 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 584/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:48:15 2022] GET /identity/v3/users/da54699c55b7426dbc4a7f033eb68db6 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:15.867661 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc4e8a-5b97-484f-873c-af30d96fbbe4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:15.937642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:15.976908 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc4e8a-5b97-484f-873c-af30d96fbbe4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:16.011955 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a98930f-ee39-4f69-b103-291952cbb303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:16.012600 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98930f-ee39-4f69-b103-291952cbb303 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:16.012884 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98930f-ee39-4f69-b103-291952cbb303 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:16.013157 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98930f-ee39-4f69-b103-291952cbb303 None admin] PATH_INFO: `/v3/projects/1e667793dbb14da0aef46a24d0efaf44` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:16.014357 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a98930f-ee39-4f69-b103-291952cbb303 None admin] RBAC: Authorizing `identity:get_project(project_id=1e667793dbb14da0aef46a24d0efaf44)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:16.015812 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a98930f-ee39-4f69-b103-291952cbb303 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:16.017511 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 568/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:48:15 2022] GET /identity/v3/projects/1e667793dbb14da0aef46a24d0efaf44 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:16.024144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c4223-b66c-4732-96e4-55a3966f7c5a None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:16.072089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:16.126988 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c4223-b66c-4732-96e4-55a3966f7c5a None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:16.162318 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29251935-3b8a-4ca4-b95a-c0c43141a9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:16.163130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29251935-3b8a-4ca4-b95a-c0c43141a9af None admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:16.163410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29251935-3b8a-4ca4-b95a-c0c43141a9af None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:16.163719 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29251935-3b8a-4ca4-b95a-c0c43141a9af None admin] PATH_INFO: `/v3/projects/1e667793dbb14da0aef46a24d0efaf44/users/da54699c55b7426dbc4a7f033eb68db6/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:16.166026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29251935-3b8a-4ca4-b95a-c0c43141a9af None admin] RBAC: Authorizing `identity:create_grant(project_id=1e667793dbb14da0aef46a24d0efaf44, user_id=da54699c55b7426dbc4a7f033eb68db6, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:16.167501 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29251935-3b8a-4ca4-b95a-c0c43141a9af None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:16.182902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 585/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jul 26 04:48:16 2022] PUT /identity/v3/projects/1e667793dbb14da0aef46a24d0efaf44/users/da54699c55b7426dbc4a7f033eb68db6/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:48:18.152480 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f07871-ddd9-4c6d-b270-442f2610facb None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:18.153045 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f07871-ddd9-4c6d-b270-442f2610facb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:18.153045 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f07871-ddd9-4c6d-b270-442f2610facb None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:18.154404 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 569/1154] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:48:18.159918 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b112118-fa2f-4bad-a51f-3b5bd54794cb None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:18.160384 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b112118-fa2f-4bad-a51f-3b5bd54794cb None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:18.160475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b112118-fa2f-4bad-a51f-3b5bd54794cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:18.179874 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-3b112118-fa2f-4bad-a51f-3b5bd54794cb None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:48:18.181084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-3b112118-fa2f-4bad-a51f-3b5bd54794cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:48:18.251284 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 586/1155] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:48:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:48:18.274532 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f07871-ddd9-4c6d-b270-442f2610facb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:18.382150 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f07871-ddd9-4c6d-b270-442f2610facb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:18.420369 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb31c15-5ff2-432f-bebe-0b52a0fe3030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:18.420944 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb31c15-5ff2-432f-bebe-0b52a0fe3030 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:18.421229 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb31c15-5ff2-432f-bebe-0b52a0fe3030 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:18.421503 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb31c15-5ff2-432f-bebe-0b52a0fe3030 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:18.423493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:18.432987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb31c15-5ff2-432f-bebe-0b52a0fe3030 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:48:18.437865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-3cb31c15-5ff2-432f-bebe-0b52a0fe3030 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:48:18.437865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:48:18.437865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:48:18.437865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:48:18.437865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:48:18.437865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:48:18.437865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:48:18.437865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:48:18.437865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:48:18.437865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:48:18.437865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:48:18.437865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:48:18.437865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:48:18.437865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:48:18.437865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:48:18.437865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:48:18.438924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:48:18.438924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:48:18.438924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:48:18.438924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:48:18.438924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:48:18.438924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:48:18.438924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:48:18.438924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:48:18.438924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:48:18.438924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:48:18.438924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:48:18.438924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:48:18.438924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:48:18.438924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:48:18.438924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:48:18.438924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:48:18.445141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:48:18.445141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:48:18.445141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:48:18.445141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:48:18.445141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:48:18.445141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:48:18.445141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:48:18.445141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:48:18.445141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:48:18.445141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:48:18.445141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:48:18.445141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:48:18.445141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 570/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:48:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:48:18.446472 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b112118-fa2f-4bad-a51f-3b5bd54794cb None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:18.479639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:18.546909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b112118-fa2f-4bad-a51f-3b5bd54794cb None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:18.581642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a77de-ea34-4356-beb7-09554177da08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:18.582266 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a77de-ea34-4356-beb7-09554177da08 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:18.582554 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a77de-ea34-4356-beb7-09554177da08 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:18.582833 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a77de-ea34-4356-beb7-09554177da08 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:18.583583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084a77de-ea34-4356-beb7-09554177da08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:48:18.583888 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084a77de-ea34-4356-beb7-09554177da08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:18.585367 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084a77de-ea34-4356-beb7-09554177da08 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:18.594085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 587/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:48:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:18.600224 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb31c15-5ff2-432f-bebe-0b52a0fe3030 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:18.719961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb31c15-5ff2-432f-bebe-0b52a0fe3030 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:18.756084 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb0fce-2d04-4bfd-80da-b9e4cd55428a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:18.756746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb0fce-2d04-4bfd-80da-b9e4cd55428a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:18.757014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb0fce-2d04-4bfd-80da-b9e4cd55428a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:18.757286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb0fce-2d04-4bfd-80da-b9e4cd55428a None admin] PATH_INFO: `/v3/users/da54699c55b7426dbc4a7f033eb68db6` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:18.758590 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb0fce-2d04-4bfd-80da-b9e4cd55428a None admin] RBAC: Authorizing `identity:get_user(user_id=da54699c55b7426dbc4a7f033eb68db6)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:18.760102 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb0fce-2d04-4bfd-80da-b9e4cd55428a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:18.761788 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 571/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:48:18 2022] GET /identity/v3/users/da54699c55b7426dbc4a7f033eb68db6 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:18.767932 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a77de-ea34-4356-beb7-09554177da08 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:18.863409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:18.870466 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a77de-ea34-4356-beb7-09554177da08 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:18.907236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad86ca-0278-4b19-8453-d16ed54c59c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:18.907867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad86ca-0278-4b19-8453-d16ed54c59c6 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:18.908146 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad86ca-0278-4b19-8453-d16ed54c59c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:18.908427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad86ca-0278-4b19-8453-d16ed54c59c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:18.980099 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad86ca-0278-4b19-8453-d16ed54c59c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:18.981536 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad86ca-0278-4b19-8453-d16ed54c59c6 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:19.076226 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:19.083261 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 588/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:48:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 04:48:19.089834 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb0fce-2d04-4bfd-80da-b9e4cd55428a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:19.154828 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:19.193732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb0fce-2d04-4bfd-80da-b9e4cd55428a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:19.230452 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f2ca9-7def-4d28-a376-7831a0c930a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:19.231022 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f2ca9-7def-4d28-a376-7831a0c930a2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:19.231307 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f2ca9-7def-4d28-a376-7831a0c930a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:19.231582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f2ca9-7def-4d28-a376-7831a0c930a2 None admin] PATH_INFO: `/v3/projects/1e667793dbb14da0aef46a24d0efaf44` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:19.232834 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f2ca9-7def-4d28-a376-7831a0c930a2 None admin] RBAC: Authorizing `identity:get_project(project_id=1e667793dbb14da0aef46a24d0efaf44)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:19.234280 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f2ca9-7def-4d28-a376-7831a0c930a2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:19.236014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 572/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:48:19 2022] GET /identity/v3/projects/1e667793dbb14da0aef46a24d0efaf44 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:19.242566 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad86ca-0278-4b19-8453-d16ed54c59c6 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:19.347734 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad86ca-0278-4b19-8453-d16ed54c59c6 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:19.382911 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e3e04c-debf-4ec1-b76c-624670d1ef30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:19.383530 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3e04c-debf-4ec1-b76c-624670d1ef30 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:19.383855 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3e04c-debf-4ec1-b76c-624670d1ef30 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:19.384131 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3e04c-debf-4ec1-b76c-624670d1ef30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:19.384930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e3e04c-debf-4ec1-b76c-624670d1ef30 None admin] RBAC: Adding query filter params (role.id=72bf8bd23d2c4b76ad32b7b35d6db412, scope.project.id=1e667793dbb14da0aef46a24d0efaf44, user.id=da54699c55b7426dbc4a7f033eb68db6) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:48:19.385245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e3e04c-debf-4ec1-b76c-624670d1ef30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:19.386706 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e3e04c-debf-4ec1-b76c-624670d1ef30 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:19.393073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 589/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:48:19 2022] GET /identity/v3/role_assignments?user.id=da54699c55b7426dbc4a7f033eb68db6&scope.project.id=1e667793dbb14da0aef46a24d0efaf44&role.id=72bf8bd23d2c4b76ad32b7b35d6db412 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:21.397798 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205d7f9-3030-4583-8562-d2e0b0a606d4 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:21.398285 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205d7f9-3030-4583-8562-d2e0b0a606d4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:21.398378 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205d7f9-3030-4583-8562-d2e0b0a606d4 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:21.399773 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 573/1162] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:48:21.405208 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf9e10-26fc-4872-9956-36c5aa4d003f None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:21.405683 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf9e10-26fc-4872-9956-36c5aa4d003f None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:21.405776 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf9e10-26fc-4872-9956-36c5aa4d003f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:21.424057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-76cf9e10-26fc-4872-9956-36c5aa4d003f None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:48:21.425261 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-76cf9e10-26fc-4872-9956-36c5aa4d003f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:48:21.480599 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:21.492291 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 590/1163] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:48:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 04:48:21.514396 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9205d7f9-3030-4583-8562-d2e0b0a606d4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:21.619202 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9205d7f9-3030-4583-8562-d2e0b0a606d4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:21.648200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:21.654858 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89652e78-4255-41f0-aa56-8fbb8bc47fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:21.655498 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89652e78-4255-41f0-aa56-8fbb8bc47fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:21.655807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89652e78-4255-41f0-aa56-8fbb8bc47fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:21.656066 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89652e78-4255-41f0-aa56-8fbb8bc47fa6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:21.724953 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89652e78-4255-41f0-aa56-8fbb8bc47fa6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:21.726403 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89652e78-4255-41f0-aa56-8fbb8bc47fa6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:21.829000 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 574/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:48:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 04:48:21.835719 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf9e10-26fc-4872-9956-36c5aa4d003f None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:21.933618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf9e10-26fc-4872-9956-36c5aa4d003f None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:21.968723 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb8267-18b1-48f9-bdc9-e8bff5048d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:21.969359 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb8267-18b1-48f9-bdc9-e8bff5048d3a None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:21.969642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb8267-18b1-48f9-bdc9-e8bff5048d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:21.969910 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb8267-18b1-48f9-bdc9-e8bff5048d3a None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:21.976935 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadb8267-18b1-48f9-bdc9-e8bff5048d3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:21.978396 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadb8267-18b1-48f9-bdc9-e8bff5048d3a None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:21.989913 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-aadb8267-18b1-48f9-bdc9-e8bff5048d3a None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 26 04:48:21.989913 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:48:21.989913 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 26 04:48:21.989913 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 26 04:48:21.989913 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:48:21.989913 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:48:21.989913 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:48:21.989913 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:48:21.989913 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 26 04:48:21.989913 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:48:21.989913 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:48:21.989913 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:48:21.989913 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:48:21.989913 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:48:21.989913 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:48:21.989913 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:48:21.989913 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:48:21.991108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:48:21.991108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:48:21.991108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:48:21.991108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:48:21.991108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:48:21.991108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:48:21.991108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:48:21.991108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:48:21.991108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:48:21.991108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:48:21.991108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:48:21.991108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:48:21.991108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:48:21.991108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:48:21.991108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:48:21.991108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:48:21.992163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:48:21.992163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:48:21.992163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:48:21.992163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:48:21.992163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:48:21.992163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:48:21.992163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:48:21.992163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:48:21.992163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:48:21.992163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:48:21.992163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 26 04:48:21.992163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:48:21.992163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 26 04:48:21.992163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:48:21.992163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 591/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Jul 26 04:48:21 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:48:21.997580 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89652e78-4255-41f0-aa56-8fbb8bc47fa6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:22.033608 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:22.115127 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89652e78-4255-41f0-aa56-8fbb8bc47fa6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:22.152200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afc7e1b-d860-4ffc-bf8b-0883edc30507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:22.152822 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc7e1b-d860-4ffc-bf8b-0883edc30507 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:22.153103 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc7e1b-d860-4ffc-bf8b-0883edc30507 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:22.153379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc7e1b-d860-4ffc-bf8b-0883edc30507 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:22.154068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afc7e1b-d860-4ffc-bf8b-0883edc30507 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:48:22.154364 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afc7e1b-d860-4ffc-bf8b-0883edc30507 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:22.155837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afc7e1b-d860-4ffc-bf8b-0883edc30507 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:22.163011 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 575/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Jul 26 04:48:21 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:24.116814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ce1e1-7e4b-406e-bfc2-d8be65c78ed5 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:24.117405 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ce1e1-7e4b-406e-bfc2-d8be65c78ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:24.117405 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ce1e1-7e4b-406e-bfc2-d8be65c78ed5 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:24.118715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 592/1167] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:48:24.124253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fe059-107d-445f-9085-596dba9057fe None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:24.124676 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fe059-107d-445f-9085-596dba9057fe None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:24.124765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fe059-107d-445f-9085-596dba9057fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:24.144374 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-620fe059-107d-445f-9085-596dba9057fe None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:48:24.145581 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-620fe059-107d-445f-9085-596dba9057fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:48:24.217152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 576/1168] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:48:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:48:24.239698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ce1e1-7e4b-406e-bfc2-d8be65c78ed5 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:24.296078 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:24.350135 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ce1e1-7e4b-406e-bfc2-d8be65c78ed5 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:24.389198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c9d6e-54c0-42b1-b715-c8d9d2505959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:24.389499 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c9d6e-54c0-42b1-b715-c8d9d2505959 None admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:24.389781 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c9d6e-54c0-42b1-b715-c8d9d2505959 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:24.390049 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c9d6e-54c0-42b1-b715-c8d9d2505959 None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:24.390765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c9d6e-54c0-42b1-b715-c8d9d2505959 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:24.392180 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c9d6e-54c0-42b1-b715-c8d9d2505959 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:24.409329 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 593/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Tue Jul 26 04:48:24 2022] POST /identity/v3/domains => generated 267 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:48:26.437637 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584536b7-2386-4568-95ed-54ef8ef84cc2 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:26.438215 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584536b7-2386-4568-95ed-54ef8ef84cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:26.438215 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584536b7-2386-4568-95ed-54ef8ef84cc2 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:26.439593 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 577/1170] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:48:26.445069 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180feb2-c460-4a61-8d7f-eb3ff723a6cd None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:26.445522 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180feb2-c460-4a61-8d7f-eb3ff723a6cd None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:26.445617 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180feb2-c460-4a61-8d7f-eb3ff723a6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:26.465134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-2180feb2-c460-4a61-8d7f-eb3ff723a6cd None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:48:26.466338 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-2180feb2-c460-4a61-8d7f-eb3ff723a6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:48:26.542498 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 594/1171] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:48:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:48:26.571590 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584536b7-2386-4568-95ed-54ef8ef84cc2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:26.598419 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:26.680652 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584536b7-2386-4568-95ed-54ef8ef84cc2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:26.715245 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a3a53-0678-406b-a39e-795847d2d1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:26.715898 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a3a53-0678-406b-a39e-795847d2d1c8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:26.716265 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a3a53-0678-406b-a39e-795847d2d1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:26.716443 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a3a53-0678-406b-a39e-795847d2d1c8 None admin] PATH_INFO: `/v3/domains/6a39d867ddf24602ad7075d9d86e42b9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:26.727039 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a3a53-0678-406b-a39e-795847d2d1c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6a39d867ddf24602ad7075d9d86e42b9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:26.728513 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a3a53-0678-406b-a39e-795847d2d1c8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:26.730146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 578/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Tue Jul 26 04:48:26 2022] GET /identity/v3/domains/6a39d867ddf24602ad7075d9d86e42b9 => generated 267 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:26.736579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180feb2-c460-4a61-8d7f-eb3ff723a6cd None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:26.780595 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:26.844340 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180feb2-c460-4a61-8d7f-eb3ff723a6cd None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:26.887144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d833f9-7d8f-4e74-8535-d3809857eb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:26.887861 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d833f9-7d8f-4e74-8535-d3809857eb88 None admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:26.888328 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d833f9-7d8f-4e74-8535-d3809857eb88 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:26.888748 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d833f9-7d8f-4e74-8535-d3809857eb88 None admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:26.889708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d833f9-7d8f-4e74-8535-d3809857eb88 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:26.891918 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d833f9-7d8f-4e74-8535-d3809857eb88 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:26.921766 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 595/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jul 26 04:48:26 2022] POST /identity/v3/projects => generated 361 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:48:28.986117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7393c-cea9-47a4-9906-2ca5c93b7d4b None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:28.986679 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7393c-cea9-47a4-9906-2ca5c93b7d4b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:28.986679 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7393c-cea9-47a4-9906-2ca5c93b7d4b None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:28.988197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 579/1174] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:48:28.994112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005b1c8-7d87-426f-9f81-1a04c2332616 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:28.994509 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005b1c8-7d87-426f-9f81-1a04c2332616 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:28.994704 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005b1c8-7d87-426f-9f81-1a04c2332616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:29.013598 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-0005b1c8-7d87-426f-9f81-1a04c2332616 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:48:29.014800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-0005b1c8-7d87-426f-9f81-1a04c2332616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:48:29.082843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 596/1175] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:48:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:48:29.108060 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d7393c-cea9-47a4-9906-2ca5c93b7d4b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:29.198775 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:29.217094 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d7393c-cea9-47a4-9906-2ca5c93b7d4b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:29.258285 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2b8cb-f7db-4a5b-b650-8c5dc5b76ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:29.259285 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2b8cb-f7db-4a5b-b650-8c5dc5b76ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:29.259838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2b8cb-f7db-4a5b-b650-8c5dc5b76ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:29.260312 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2b8cb-f7db-4a5b-b650-8c5dc5b76ab4 None admin] PATH_INFO: `/v3/domains/6a39d867ddf24602ad7075d9d86e42b9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:29.262300 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf2b8cb-f7db-4a5b-b650-8c5dc5b76ab4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6a39d867ddf24602ad7075d9d86e42b9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:29.264584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf2b8cb-f7db-4a5b-b650-8c5dc5b76ab4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:29.266284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 580/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Tue Jul 26 04:48:29 2022] GET /identity/v3/domains/6a39d867ddf24602ad7075d9d86e42b9 => generated 267 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:29.272950 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005b1c8-7d87-426f-9f81-1a04c2332616 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:29.300739 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:29.382520 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005b1c8-7d87-426f-9f81-1a04c2332616 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:29.417815 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d8e51c-d999-4c75-a53c-33be5a9a0a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:29.418604 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8e51c-d999-4c75-a53c-33be5a9a0a95 None admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:29.418859 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8e51c-d999-4c75-a53c-33be5a9a0a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:29.419252 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8e51c-d999-4c75-a53c-33be5a9a0a95 None admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:29.419969 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d8e51c-d999-4c75-a53c-33be5a9a0a95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:29.421469 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d8e51c-d999-4c75-a53c-33be5a9a0a95 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:29.443744 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 597/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jul 26 04:48:29 2022] POST /identity/v3/users => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:48:31.469130 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be59597-0e32-487e-b699-d75498f3d18a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:31.469724 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be59597-0e32-487e-b699-d75498f3d18a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:31.469724 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be59597-0e32-487e-b699-d75498f3d18a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:31.471027 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 581/1178] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:48:31.476505 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e853b1b-fecc-4e8f-8b7d-ea94095d99e7 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:31.476967 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e853b1b-fecc-4e8f-8b7d-ea94095d99e7 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:31.477073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e853b1b-fecc-4e8f-8b7d-ea94095d99e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:31.496964 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-0e853b1b-fecc-4e8f-8b7d-ea94095d99e7 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:48:31.498191 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-0e853b1b-fecc-4e8f-8b7d-ea94095d99e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:48:31.569555 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 598/1179] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:48:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:48:31.593334 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be59597-0e32-487e-b699-d75498f3d18a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:31.699600 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be59597-0e32-487e-b699-d75498f3d18a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:31.734717 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c533a-16b2-44d4-8b59-135ba8f6a311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:31.735350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c533a-16b2-44d4-8b59-135ba8f6a311 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:31.735656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c533a-16b2-44d4-8b59-135ba8f6a311 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:31.735924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c533a-16b2-44d4-8b59-135ba8f6a311 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:31.746714 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c533a-16b2-44d4-8b59-135ba8f6a311 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:48:31.751485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-6a6c533a-16b2-44d4-8b59-135ba8f6a311 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:48:31.751485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:48:31.751485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:48:31.751485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:48:31.751485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:48:31.751485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:48:31.751485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:48:31.751485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:48:31.751485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:48:31.751485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:48:31.751485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:48:31.751485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:48:31.751485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:48:31.751485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:48:31.751485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:48:31.751485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:48:31.752656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:48:31.752656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:48:31.752656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:48:31.752656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:48:31.752656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:48:31.752656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:48:31.752656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:48:31.752656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:48:31.752656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:48:31.752656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:48:31.752656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:48:31.752656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:48:31.752656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:48:31.752656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:48:31.752656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:48:31.752656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:48:31.753594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:48:31.753594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:48:31.753594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:48:31.753594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:48:31.753594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:48:31.753594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:48:31.753594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:48:31.753594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:48:31.753594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:48:31.753594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:48:31.753594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:48:31.753594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:48:31.753594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 582/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:48:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:48:31.759489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e853b1b-fecc-4e8f-8b7d-ea94095d99e7 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:31.770261 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:31.865692 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e853b1b-fecc-4e8f-8b7d-ea94095d99e7 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:31.901301 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a5508-f48b-4c7b-9a5e-6d237257e4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:31.901915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a5508-f48b-4c7b-9a5e-6d237257e4e4 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:31.902190 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a5508-f48b-4c7b-9a5e-6d237257e4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:31.902457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a5508-f48b-4c7b-9a5e-6d237257e4e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:31.903175 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a5508-f48b-4c7b-9a5e-6d237257e4e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:48:31.903478 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a5508-f48b-4c7b-9a5e-6d237257e4e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:31.904888 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a5508-f48b-4c7b-9a5e-6d237257e4e4 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:31.915251 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 599/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:48:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:31.921385 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c533a-16b2-44d4-8b59-135ba8f6a311 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:31.937180 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:32.025628 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c533a-16b2-44d4-8b59-135ba8f6a311 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:32.061374 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa515cbd-1900-4ef1-aee6-ad131e2ad589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:32.062018 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa515cbd-1900-4ef1-aee6-ad131e2ad589 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:32.062296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa515cbd-1900-4ef1-aee6-ad131e2ad589 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:32.062570 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa515cbd-1900-4ef1-aee6-ad131e2ad589 None admin] PATH_INFO: `/v3/users/3a5accd341234bac9c4ff382db1ce8fa` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:32.075649 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa515cbd-1900-4ef1-aee6-ad131e2ad589 None admin] RBAC: Authorizing `identity:get_user(user_id=3a5accd341234bac9c4ff382db1ce8fa)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:32.077099 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa515cbd-1900-4ef1-aee6-ad131e2ad589 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:32.078729 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 583/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:48:31 2022] GET /identity/v3/users/3a5accd341234bac9c4ff382db1ce8fa => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:32.085139 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a5508-f48b-4c7b-9a5e-6d237257e4e4 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:32.161266 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:32.193006 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a5508-f48b-4c7b-9a5e-6d237257e4e4 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:32.228571 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee410f-1473-4e62-af5e-f5b35c3d4ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:32.229192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee410f-1473-4e62-af5e-f5b35c3d4ede None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:32.229476 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee410f-1473-4e62-af5e-f5b35c3d4ede None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:32.229747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee410f-1473-4e62-af5e-f5b35c3d4ede None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:32.299879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee410f-1473-4e62-af5e-f5b35c3d4ede None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:32.301264 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee410f-1473-4e62-af5e-f5b35c3d4ede None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:32.370848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:32.404067 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 600/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:48:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 04:48:32.410330 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa515cbd-1900-4ef1-aee6-ad131e2ad589 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:32.482255 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:32.514072 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa515cbd-1900-4ef1-aee6-ad131e2ad589 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:32.550849 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8a828-0068-4ee6-acbe-74e5a0fcd0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:32.552015 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8a828-0068-4ee6-acbe-74e5a0fcd0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:32.552301 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8a828-0068-4ee6-acbe-74e5a0fcd0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:32.552600 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8a828-0068-4ee6-acbe-74e5a0fcd0c9 None admin] PATH_INFO: `/v3/projects/5be740bd24344dd1af7552ef6ff64c43` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:32.553890 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8a828-0068-4ee6-acbe-74e5a0fcd0c9 None admin] RBAC: Authorizing `identity:get_project(project_id=5be740bd24344dd1af7552ef6ff64c43)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:32.555373 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8a828-0068-4ee6-acbe-74e5a0fcd0c9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:32.557144 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 584/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:48:32 2022] GET /identity/v3/projects/5be740bd24344dd1af7552ef6ff64c43 => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:32.563587 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee410f-1473-4e62-af5e-f5b35c3d4ede None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:32.667863 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee410f-1473-4e62-af5e-f5b35c3d4ede None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:32.704016 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d120616-0273-4a04-8247-c4a35d29f791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:32.704616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d120616-0273-4a04-8247-c4a35d29f791 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:32.704908 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d120616-0273-4a04-8247-c4a35d29f791 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:32.705176 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d120616-0273-4a04-8247-c4a35d29f791 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:32.705980 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d120616-0273-4a04-8247-c4a35d29f791 None admin] RBAC: Adding query filter params (role.id=72bf8bd23d2c4b76ad32b7b35d6db412, scope.project.id=5be740bd24344dd1af7552ef6ff64c43, user.id=3a5accd341234bac9c4ff382db1ce8fa) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:48:32.706274 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d120616-0273-4a04-8247-c4a35d29f791 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:32.707735 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d120616-0273-4a04-8247-c4a35d29f791 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:32.713701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 601/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:48:32 2022] GET /identity/v3/role_assignments?user.id=3a5accd341234bac9c4ff382db1ce8fa&scope.project.id=5be740bd24344dd1af7552ef6ff64c43&role.id=72bf8bd23d2c4b76ad32b7b35d6db412 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:34.656922 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5ef99-8957-4878-b0b9-4bb465950a6e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:34.657552 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5ef99-8957-4878-b0b9-4bb465950a6e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:34.657552 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5ef99-8957-4878-b0b9-4bb465950a6e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:34.658829 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 585/1186] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:48:34.664305 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5492f6f-eaab-4c5c-8e23-5c0662d4787c None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:34.664843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5492f6f-eaab-4c5c-8e23-5c0662d4787c None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:34.664843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5492f6f-eaab-4c5c-8e23-5c0662d4787c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:34.684394 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-c5492f6f-eaab-4c5c-8e23-5c0662d4787c None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:48:34.685597 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c5492f6f-eaab-4c5c-8e23-5c0662d4787c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:48:34.714682 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:34.756642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 602/1187] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:48:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:48:34.778647 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5ef99-8957-4878-b0b9-4bb465950a6e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:34.896844 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5ef99-8957-4878-b0b9-4bb465950a6e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:34.933666 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cdd1c2-ff53-4fdb-88ee-d0e6e1e7b85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:34.934310 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdd1c2-ff53-4fdb-88ee-d0e6e1e7b85f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:34.934648 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdd1c2-ff53-4fdb-88ee-d0e6e1e7b85f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:34.935129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdd1c2-ff53-4fdb-88ee-d0e6e1e7b85f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:34.942734 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:34.947118 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cdd1c2-ff53-4fdb-88ee-d0e6e1e7b85f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:48:34.952309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c4cdd1c2-ff53-4fdb-88ee-d0e6e1e7b85f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:48:34.952309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:48:34.952309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:48:34.952309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:48:34.952309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:48:34.952309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:48:34.952309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:48:34.952309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:48:34.952309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:48:34.952309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:48:34.952309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:48:34.952309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:48:34.952309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:48:34.952309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:48:34.952309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:48:34.952309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:48:34.953427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:48:34.953427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:48:34.953427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:48:34.953427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:48:34.953427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:48:34.953427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:48:34.953427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:48:34.953427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:48:34.953427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:48:34.953427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:48:34.953427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:48:34.953427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:48:34.953427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:48:34.953427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:48:34.953427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:48:34.953427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:48:34.954665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:48:34.954665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:48:34.954665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:48:34.954665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:48:34.954665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:48:34.954665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:48:34.954665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:48:34.954665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:48:34.954665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:48:34.954665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:48:34.954665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:48:34.954665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:48:34.954665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 586/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:48:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:48:34.961268 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5492f6f-eaab-4c5c-8e23-5c0662d4787c None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:35.072891 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5492f6f-eaab-4c5c-8e23-5c0662d4787c None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:35.109198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a52e8-99e1-4168-8064-1ee8131362e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:35.109821 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a52e8-99e1-4168-8064-1ee8131362e9 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:35.110098 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a52e8-99e1-4168-8064-1ee8131362e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:35.110365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a52e8-99e1-4168-8064-1ee8131362e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:35.111083 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a52e8-99e1-4168-8064-1ee8131362e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:48:35.111389 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a52e8-99e1-4168-8064-1ee8131362e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:35.112814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a52e8-99e1-4168-8064-1ee8131362e9 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:35.121566 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 603/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:48:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:35.130173 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cdd1c2-ff53-4fdb-88ee-d0e6e1e7b85f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:35.239714 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cdd1c2-ff53-4fdb-88ee-d0e6e1e7b85f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:35.283529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6749e7-0076-4315-9227-863de21ce0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:35.284459 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6749e7-0076-4315-9227-863de21ce0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:35.284759 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6749e7-0076-4315-9227-863de21ce0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:35.285036 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6749e7-0076-4315-9227-863de21ce0c7 None admin] PATH_INFO: `/v3/users/3a5accd341234bac9c4ff382db1ce8fa` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:35.287027 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6749e7-0076-4315-9227-863de21ce0c7 None admin] RBAC: Authorizing `identity:get_user(user_id=3a5accd341234bac9c4ff382db1ce8fa)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:35.289272 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6749e7-0076-4315-9227-863de21ce0c7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:35.291630 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 587/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:48:35 2022] GET /identity/v3/users/3a5accd341234bac9c4ff382db1ce8fa => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:35.299481 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a52e8-99e1-4168-8064-1ee8131362e9 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:35.328799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:35.411894 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a52e8-99e1-4168-8064-1ee8131362e9 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:35.446941 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c7644-d009-4dd3-8106-05d95fdd5f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:35.447593 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c7644-d009-4dd3-8106-05d95fdd5f1e None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:35.447952 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c7644-d009-4dd3-8106-05d95fdd5f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:35.448233 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c7644-d009-4dd3-8106-05d95fdd5f1e None admin] PATH_INFO: `/v3/projects/5be740bd24344dd1af7552ef6ff64c43` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:35.449409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06c7644-d009-4dd3-8106-05d95fdd5f1e None admin] RBAC: Authorizing `identity:get_project(project_id=5be740bd24344dd1af7552ef6ff64c43)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:35.450902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06c7644-d009-4dd3-8106-05d95fdd5f1e None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:35.452647 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 604/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:48:35 2022] GET /identity/v3/projects/5be740bd24344dd1af7552ef6ff64c43 => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:35.458887 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6749e7-0076-4315-9227-863de21ce0c7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:35.526508 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:35.565469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6749e7-0076-4315-9227-863de21ce0c7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:35.602315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f1d1e-a1a2-44c6-9ae8-ee048ad92517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:35.603022 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f1d1e-a1a2-44c6-9ae8-ee048ad92517 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:35.603306 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f1d1e-a1a2-44c6-9ae8-ee048ad92517 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:35.603604 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f1d1e-a1a2-44c6-9ae8-ee048ad92517 None admin] PATH_INFO: `/v3/projects/5be740bd24344dd1af7552ef6ff64c43/users/3a5accd341234bac9c4ff382db1ce8fa/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:35.605543 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f1d1e-a1a2-44c6-9ae8-ee048ad92517 None admin] RBAC: Authorizing `identity:create_grant(project_id=5be740bd24344dd1af7552ef6ff64c43, user_id=3a5accd341234bac9c4ff382db1ce8fa, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:35.606985 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f1d1e-a1a2-44c6-9ae8-ee048ad92517 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:35.621600 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 588/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jul 26 04:48:35 2022] PUT /identity/v3/projects/5be740bd24344dd1af7552ef6ff64c43/users/3a5accd341234bac9c4ff382db1ce8fa/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:48:37.613034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ae2c1-621e-4f6e-9085-65887fe11fc6 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:37.613629 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ae2c1-621e-4f6e-9085-65887fe11fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:37.613629 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ae2c1-621e-4f6e-9085-65887fe11fc6 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:37.614909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 605/1193] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:48:37.620607 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fbf968-4026-4544-954f-715baae2890c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:37.621048 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fbf968-4026-4544-954f-715baae2890c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:37.621048 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fbf968-4026-4544-954f-715baae2890c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:37.641041 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c7fbf968-4026-4544-954f-715baae2890c None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:48:37.642198 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c7fbf968-4026-4544-954f-715baae2890c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:48:37.714633 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 589/1194] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:48:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:48:37.738342 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ae2c1-621e-4f6e-9085-65887fe11fc6 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:37.845141 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:37.853163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ae2c1-621e-4f6e-9085-65887fe11fc6 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:37.888337 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979abc3c-79d8-42b0-8d5a-8c5f2d0f4d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:37.889003 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979abc3c-79d8-42b0-8d5a-8c5f2d0f4d7f None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:37.889256 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979abc3c-79d8-42b0-8d5a-8c5f2d0f4d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:37.889554 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979abc3c-79d8-42b0-8d5a-8c5f2d0f4d7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:37.900366 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979abc3c-79d8-42b0-8d5a-8c5f2d0f4d7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:48:37.905119 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-979abc3c-79d8-42b0-8d5a-8c5f2d0f4d7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:48:37.905119 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:48:37.905119 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:48:37.905119 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:48:37.905119 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:48:37.905119 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:48:37.905119 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:48:37.905119 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:48:37.905119 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:48:37.905119 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:48:37.905119 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:48:37.905119 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:48:37.905119 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:48:37.905119 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:48:37.905119 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:48:37.905119 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:48:37.906220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:48:37.906220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:48:37.906220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:48:37.906220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:48:37.906220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:48:37.906220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:48:37.906220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:48:37.906220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:48:37.906220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:48:37.906220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:48:37.906220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:48:37.906220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:48:37.906220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:48:37.906220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:48:37.906220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:48:37.906220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:48:37.907144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:48:37.907144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:48:37.907144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:48:37.907144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:48:37.907144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:48:37.907144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:48:37.907144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:48:37.907144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:48:37.907144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:48:37.907144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:48:37.907144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:48:37.907144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:48:37.907144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 606/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:48:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:48:37.912993 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fbf968-4026-4544-954f-715baae2890c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:37.973135 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:38.019834 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fbf968-4026-4544-954f-715baae2890c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:38.056324 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492f653-a69f-44a0-b8b6-adac9c1b0e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:38.056958 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492f653-a69f-44a0-b8b6-adac9c1b0e1d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:38.057241 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492f653-a69f-44a0-b8b6-adac9c1b0e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:38.057516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492f653-a69f-44a0-b8b6-adac9c1b0e1d None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:38.058242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492f653-a69f-44a0-b8b6-adac9c1b0e1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:48:38.058548 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492f653-a69f-44a0-b8b6-adac9c1b0e1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:38.059973 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492f653-a69f-44a0-b8b6-adac9c1b0e1d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:38.068458 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 590/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:48:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:38.074584 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979abc3c-79d8-42b0-8d5a-8c5f2d0f4d7f None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:38.191760 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979abc3c-79d8-42b0-8d5a-8c5f2d0f4d7f None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:38.227034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b77fbe-8568-476e-96fd-fbe62f501cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:38.227708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b77fbe-8568-476e-96fd-fbe62f501cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:38.227991 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b77fbe-8568-476e-96fd-fbe62f501cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:38.228261 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b77fbe-8568-476e-96fd-fbe62f501cd4 None admin] PATH_INFO: `/v3/users/3a5accd341234bac9c4ff382db1ce8fa` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:38.229435 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b77fbe-8568-476e-96fd-fbe62f501cd4 None admin] RBAC: Authorizing `identity:get_user(user_id=3a5accd341234bac9c4ff382db1ce8fa)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:38.230879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b77fbe-8568-476e-96fd-fbe62f501cd4 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:38.232548 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 607/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jul 26 04:48:38 2022] GET /identity/v3/users/3a5accd341234bac9c4ff382db1ce8fa => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:38.238750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492f653-a69f-44a0-b8b6-adac9c1b0e1d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:38.340530 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492f653-a69f-44a0-b8b6-adac9c1b0e1d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:38.357615 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:38.374592 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4fd3af-9e20-47f4-8b7f-2be9f1d8ec10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:38.375215 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4fd3af-9e20-47f4-8b7f-2be9f1d8ec10 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:38.375524 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4fd3af-9e20-47f4-8b7f-2be9f1d8ec10 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:38.375803 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4fd3af-9e20-47f4-8b7f-2be9f1d8ec10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:38.445919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4fd3af-9e20-47f4-8b7f-2be9f1d8ec10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:38.447301 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4fd3af-9e20-47f4-8b7f-2be9f1d8ec10 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:38.558548 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 591/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:48:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 04:48:38.564773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b77fbe-8568-476e-96fd-fbe62f501cd4 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:38.592466 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:38.672010 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b77fbe-8568-476e-96fd-fbe62f501cd4 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:38.706620 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ee0c3-e463-4dad-b86a-9c275640be74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:38.707182 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ee0c3-e463-4dad-b86a-9c275640be74 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:38.707463 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ee0c3-e463-4dad-b86a-9c275640be74 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:38.707748 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ee0c3-e463-4dad-b86a-9c275640be74 None admin] PATH_INFO: `/v3/projects/5be740bd24344dd1af7552ef6ff64c43` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:38.709043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3ee0c3-e463-4dad-b86a-9c275640be74 None admin] RBAC: Authorizing `identity:get_project(project_id=5be740bd24344dd1af7552ef6ff64c43)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:38.710858 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3ee0c3-e463-4dad-b86a-9c275640be74 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:38.712612 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 608/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jul 26 04:48:38 2022] GET /identity/v3/projects/5be740bd24344dd1af7552ef6ff64c43 => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:38.719050 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4fd3af-9e20-47f4-8b7f-2be9f1d8ec10 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:48:38.737997 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:38.822350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4fd3af-9e20-47f4-8b7f-2be9f1d8ec10 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:48:38.858291 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6b2f6-edb0-4936-833b-44e1b62cea85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:48:38.858912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6b2f6-edb0-4936-833b-44e1b62cea85 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:48:38.859199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6b2f6-edb0-4936-833b-44e1b62cea85 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:48:38.859479 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6b2f6-edb0-4936-833b-44e1b62cea85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:48:38.860308 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6b2f6-edb0-4936-833b-44e1b62cea85 None admin] RBAC: Adding query filter params (role.id=72bf8bd23d2c4b76ad32b7b35d6db412, scope.project.id=5be740bd24344dd1af7552ef6ff64c43, user.id=3a5accd341234bac9c4ff382db1ce8fa) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:48:38.860622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6b2f6-edb0-4936-833b-44e1b62cea85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:48:38.862070 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6b2f6-edb0-4936-833b-44e1b62cea85 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:48:38.868135 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 592/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:48:38 2022] GET /identity/v3/role_assignments?user.id=3a5accd341234bac9c4ff382db1ce8fa&scope.project.id=5be740bd24344dd1af7552ef6ff64c43&role.id=72bf8bd23d2c4b76ad32b7b35d6db412 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:48:48.702713 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=55 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:48:48.865620 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=59 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:34.438966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e14f3f-0559-4c68-b9d4-6cb99978ad8b None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:34.439626 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e14f3f-0559-4c68-b9d4-6cb99978ad8b None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:34.439626 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e14f3f-0559-4c68-b9d4-6cb99978ad8b None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:34.441004 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 609/1201] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:49:34.446802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c9758-c129-42ab-98f9-ce887666a629 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:34.447283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c9758-c129-42ab-98f9-ce887666a629 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:34.447368 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c9758-c129-42ab-98f9-ce887666a629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:34.468061 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c39c9758-c129-42ab-98f9-ce887666a629 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:49:34.469407 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c39c9758-c129-42ab-98f9-ce887666a629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:49:34.559107 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 593/1202] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:49:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:49:34.594889 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e14f3f-0559-4c68-b9d4-6cb99978ad8b None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:34.722575 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e14f3f-0559-4c68-b9d4-6cb99978ad8b None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:34.779046 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3bba34-f59f-49e7-a949-a5bc7ad34ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:34.779766 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3bba34-f59f-49e7-a949-a5bc7ad34ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:34.780052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3bba34-f59f-49e7-a949-a5bc7ad34ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:34.780326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3bba34-f59f-49e7-a949-a5bc7ad34ea8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:34.795307 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3bba34-f59f-49e7-a949-a5bc7ad34ea8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:34.796867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3bba34-f59f-49e7-a949-a5bc7ad34ea8 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:34.817523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-9e3bba34-f59f-49e7-a949-a5bc7ad34ea8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:49:34.817523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:49:34.817523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:49:34.817523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:49:34.817523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:49:34.817523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:49:34.817523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:49:34.817523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:49:34.817523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:49:34.817523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:49:34.817523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:49:34.817523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:49:34.817523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:49:34.817523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:49:34.817523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:49:34.818652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:49:34.818652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:49:34.818652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:49:34.818652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:49:34.818652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:49:34.818652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:49:34.818652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:49:34.818652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:49:34.818652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:49:34.818652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:49:34.818652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:49:34.818652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:49:34.818652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:49:34.818652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:49:34.818652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:49:34.818652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:49:34.818652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:49:34.825027 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:49:34.825027 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:49:34.825027 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:49:34.825027 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:49:34.825027 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 610/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:49:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:49:34.826114 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c9758-c129-42ab-98f9-ce887666a629 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:34.970477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c9758-c129-42ab-98f9-ce887666a629 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:35.015984 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:35.023904 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140e355-09a7-4c77-ad51-27c6371a331f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:35.024598 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140e355-09a7-4c77-ad51-27c6371a331f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:35.024884 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140e355-09a7-4c77-ad51-27c6371a331f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:35.025160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140e355-09a7-4c77-ad51-27c6371a331f None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:35.025891 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140e355-09a7-4c77-ad51-27c6371a331f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:35.026192 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140e355-09a7-4c77-ad51-27c6371a331f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:35.027622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140e355-09a7-4c77-ad51-27c6371a331f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:35.041576 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 594/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:49:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:49:35.047659 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3bba34-f59f-49e7-a949-a5bc7ad34ea8 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:35.115268 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:35.180491 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3bba34-f59f-49e7-a949-a5bc7ad34ea8 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:35.227353 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3de1b6-bbb2-4d6f-800c-8037066bde22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:35.228438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3de1b6-bbb2-4d6f-800c-8037066bde22 None admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:35.228915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3de1b6-bbb2-4d6f-800c-8037066bde22 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:35.229314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3de1b6-bbb2-4d6f-800c-8037066bde22 None admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:35.230215 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3de1b6-bbb2-4d6f-800c-8037066bde22 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:35.231855 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3de1b6-bbb2-4d6f-800c-8037066bde22 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:35.259150 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 611/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jul 26 04:49:35 2022] POST /identity/v3/users => generated 251 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 04:49:37.480999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10be482-b042-4573-a3a2-e5c272eee80e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:37.481572 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10be482-b042-4573-a3a2-e5c272eee80e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:37.481572 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10be482-b042-4573-a3a2-e5c272eee80e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:37.483403 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 595/1206] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:49:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:49:37.489248 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c998c5-a48f-4e45-ad66-b1ee8685f2b9 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:37.489740 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c998c5-a48f-4e45-ad66-b1ee8685f2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:37.489820 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c998c5-a48f-4e45-ad66-b1ee8685f2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:37.511450 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-33c998c5-a48f-4e45-ad66-b1ee8685f2b9 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:49:37.512956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-33c998c5-a48f-4e45-ad66-b1ee8685f2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:49:37.593166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 612/1207] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:49:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:49:37.616694 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10be482-b042-4573-a3a2-e5c272eee80e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:37.746191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10be482-b042-4573-a3a2-e5c272eee80e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:37.801883 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65421743-2bcc-45e0-8f8b-3b63802a319e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:37.802749 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65421743-2bcc-45e0-8f8b-3b63802a319e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:37.802950 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65421743-2bcc-45e0-8f8b-3b63802a319e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:37.803260 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65421743-2bcc-45e0-8f8b-3b63802a319e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:37.814967 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65421743-2bcc-45e0-8f8b-3b63802a319e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:49:37.819919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-65421743-2bcc-45e0-8f8b-3b63802a319e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:49:37.819919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:49:37.819919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:49:37.819919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:49:37.819919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:49:37.819919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:49:37.819919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:49:37.819919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:49:37.819919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:49:37.819919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:49:37.819919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:49:37.819919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:49:37.819919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:49:37.819919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:49:37.819919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:49:37.819919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:49:37.821242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:49:37.821242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:49:37.821242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:49:37.821242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:49:37.821242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:49:37.821242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:49:37.821242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:49:37.821242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:49:37.821242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:49:37.821242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:49:37.821242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:49:37.821242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:49:37.821242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:49:37.821242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:49:37.821242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:49:37.821242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:49:37.822303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:49:37.822303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:49:37.822303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:49:37.822303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:49:37.822303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:49:37.822303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:49:37.822303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:49:37.822303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:49:37.822303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:49:37.822303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:49:37.822303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:49:37.822303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:49:37.823755 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 596/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jul 26 04:49:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:49:37.828147 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c998c5-a48f-4e45-ad66-b1ee8685f2b9 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:37.879361 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:37.954652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c998c5-a48f-4e45-ad66-b1ee8685f2b9 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:37.995310 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3af891-a1b6-41ba-8f7c-700dd1fa11d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:37.996280 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3af891-a1b6-41ba-8f7c-700dd1fa11d4 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:37.996653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3af891-a1b6-41ba-8f7c-700dd1fa11d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:37.997051 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3af891-a1b6-41ba-8f7c-700dd1fa11d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:37.997969 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3af891-a1b6-41ba-8f7c-700dd1fa11d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:37.998370 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3af891-a1b6-41ba-8f7c-700dd1fa11d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:38.000029 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3af891-a1b6-41ba-8f7c-700dd1fa11d4 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:38.016661 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 613/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jul 26 04:49:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:49:38.024470 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65421743-2bcc-45e0-8f8b-3b63802a319e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:38.090031 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:38.149744 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65421743-2bcc-45e0-8f8b-3b63802a319e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:38.193625 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0327a12-8ca7-44f4-90fb-3430450ed2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:38.194437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0327a12-8ca7-44f4-90fb-3430450ed2eb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:38.194758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0327a12-8ca7-44f4-90fb-3430450ed2eb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:38.195008 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0327a12-8ca7-44f4-90fb-3430450ed2eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:38.208284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0327a12-8ca7-44f4-90fb-3430450ed2eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:38.209919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0327a12-8ca7-44f4-90fb-3430450ed2eb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:38.226320 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e0327a12-8ca7-44f4-90fb-3430450ed2eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:49:38.226320 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:49:38.226320 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:49:38.226320 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:49:38.226320 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:49:38.226320 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:49:38.226320 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:49:38.226320 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:49:38.226320 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:49:38.226320 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:49:38.226320 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:49:38.226320 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:49:38.226320 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:49:38.226320 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:49:38.226320 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:49:38.237160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:49:38.237160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:49:38.237160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:49:38.237160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:49:38.237160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:49:38.237160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:49:38.237160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:49:38.237160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:49:38.237160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:49:38.237160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:49:38.237160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:49:38.237160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:49:38.237160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:49:38.237160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:49:38.237160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:49:38.237160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:49:38.237160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:49:38.238389 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:49:38.238389 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:49:38.238389 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:49:38.238389 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:49:38.238389 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 597/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:49:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:49:38.238726 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3af891-a1b6-41ba-8f7c-700dd1fa11d4 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:38.356100 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:38.359596 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3af891-a1b6-41ba-8f7c-700dd1fa11d4 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:38.402666 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed357152-71e5-47fc-b225-5505ab59f8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:38.403375 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed357152-71e5-47fc-b225-5505ab59f8d2 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:38.403685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed357152-71e5-47fc-b225-5505ab59f8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:38.403959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed357152-71e5-47fc-b225-5505ab59f8d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:38.404682 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed357152-71e5-47fc-b225-5505ab59f8d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:38.405174 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed357152-71e5-47fc-b225-5505ab59f8d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:38.406444 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed357152-71e5-47fc-b225-5505ab59f8d2 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:38.420112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 614/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:49:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:49:38.426083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0327a12-8ca7-44f4-90fb-3430450ed2eb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:38.541712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0327a12-8ca7-44f4-90fb-3430450ed2eb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:38.543439 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:38.581770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f85cf5-1c51-4c4c-9327-ee4d50287229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:38.582915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f85cf5-1c51-4c4c-9327-ee4d50287229 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:38.582915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f85cf5-1c51-4c4c-9327-ee4d50287229 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:38.583111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f85cf5-1c51-4c4c-9327-ee4d50287229 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:38.591416 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f85cf5-1c51-4c4c-9327-ee4d50287229 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:38.592977 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f85cf5-1c51-4c4c-9327-ee4d50287229 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:38.605912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-98f85cf5-1c51-4c4c-9327-ee4d50287229 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 04:49:38.605912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:49:38.605912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:49:38.605912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:49:38.605912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:49:38.605912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:49:38.605912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:49:38.605912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:49:38.605912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:49:38.605912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:49:38.605912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:49:38.605912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:49:38.605912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:49:38.605912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:49:38.605912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:49:38.605912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:49:38.607423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:49:38.607423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:49:38.607423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:49:38.607423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:49:38.607423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:49:38.607423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:49:38.607423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:49:38.607423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:49:38.607423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:49:38.607423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:49:38.607423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:49:38.607423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:49:38.607423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:49:38.607423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:49:38.607423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:49:38.607423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:49:38.607423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:49:38.607423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:49:38.608717 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:49:38.608717 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:49:38.608717 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:49:38.608717 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:49:38.608717 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:49:38.608717 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:49:38.608717 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:49:38.608717 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:49:38.608717 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 04:49:38.608717 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:49:38.608717 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 598/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:49:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:49:38.613096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed357152-71e5-47fc-b225-5505ab59f8d2 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:38.737326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed357152-71e5-47fc-b225-5505ab59f8d2 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:38.775829 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d79aa-0645-4613-b348-5ca714e85fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:38.776466 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d79aa-0645-4613-b348-5ca714e85fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:38.776749 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d79aa-0645-4613-b348-5ca714e85fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:38.777020 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d79aa-0645-4613-b348-5ca714e85fa7 None admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:38.777831 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d79aa-0645-4613-b348-5ca714e85fa7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:38.778139 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d79aa-0645-4613-b348-5ca714e85fa7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:38.779632 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d79aa-0645-4613-b348-5ca714e85fa7 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:38.800102 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 615/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Jul 26 04:49:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:49:38.806420 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f85cf5-1c51-4c4c-9327-ee4d50287229 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:38.937225 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f85cf5-1c51-4c4c-9327-ee4d50287229 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:38.983942 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0435928e-4dfb-445b-b838-415e25ce0c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:38.984636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435928e-4dfb-445b-b838-415e25ce0c22 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:38.985101 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435928e-4dfb-445b-b838-415e25ce0c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:38.985381 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435928e-4dfb-445b-b838-415e25ce0c22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:39.020974 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:39.089004 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0435928e-4dfb-445b-b838-415e25ce0c22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:39.091516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0435928e-4dfb-445b-b838-415e25ce0c22 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:39.223083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 599/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:49:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 04:49:39.231394 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d79aa-0645-4613-b348-5ca714e85fa7 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:39.257874 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:39.354298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d79aa-0645-4613-b348-5ca714e85fa7 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:39.393336 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ba84b-970e-4a76-aa5f-e1adafd4e9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:39.394229 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ba84b-970e-4a76-aa5f-e1adafd4e9a4 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:39.394506 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ba84b-970e-4a76-aa5f-e1adafd4e9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:39.394769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ba84b-970e-4a76-aa5f-e1adafd4e9a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:39.410267 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ba84b-970e-4a76-aa5f-e1adafd4e9a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:39.412173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ba84b-970e-4a76-aa5f-e1adafd4e9a4 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:39.428949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-543ba84b-970e-4a76-aa5f-e1adafd4e9a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:49:39.428949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:49:39.428949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:49:39.428949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:49:39.428949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:49:39.428949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:49:39.428949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:49:39.428949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:49:39.428949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:49:39.428949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:49:39.428949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:49:39.428949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:49:39.428949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:49:39.428949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:49:39.428949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:49:39.430009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:49:39.430009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:49:39.430009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:49:39.430009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:49:39.430009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:49:39.430009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:49:39.430009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:49:39.430009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:49:39.430009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:49:39.430009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:49:39.430009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:49:39.430009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:49:39.430009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:49:39.430009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:49:39.430009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:49:39.430009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:49:39.430009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:49:39.431179 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:49:39.431179 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:49:39.431179 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:49:39.431179 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:49:39.431179 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 616/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:49:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:49:39.437104 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0435928e-4dfb-445b-b838-415e25ce0c22 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:39.511037 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:39.558299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0435928e-4dfb-445b-b838-415e25ce0c22 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:39.596785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba8c40-1824-4c1c-9687-7b78923d7c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:39.597479 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba8c40-1824-4c1c-9687-7b78923d7c0c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:39.597729 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba8c40-1824-4c1c-9687-7b78923d7c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:39.597995 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba8c40-1824-4c1c-9687-7b78923d7c0c None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:39.598665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba8c40-1824-4c1c-9687-7b78923d7c0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:39.598964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba8c40-1824-4c1c-9687-7b78923d7c0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:39.600523 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba8c40-1824-4c1c-9687-7b78923d7c0c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:39.613536 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 600/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:49:39.619110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ba84b-970e-4a76-aa5f-e1adafd4e9a4 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:39.695798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:39.732802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ba84b-970e-4a76-aa5f-e1adafd4e9a4 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:39.772557 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590c49d-fcd7-4f70-90ba-53f6376b98d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:39.773185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590c49d-fcd7-4f70-90ba-53f6376b98d5 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:39.773466 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590c49d-fcd7-4f70-90ba-53f6376b98d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:39.773729 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590c49d-fcd7-4f70-90ba-53f6376b98d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:39.780943 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590c49d-fcd7-4f70-90ba-53f6376b98d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:39.782434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590c49d-fcd7-4f70-90ba-53f6376b98d5 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:39.796455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-f590c49d-fcd7-4f70-90ba-53f6376b98d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:49:39.796455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:49:39.796455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:49:39.796455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:49:39.796455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:49:39.796455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:49:39.796455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:49:39.796455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:49:39.796455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:49:39.796455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:49:39.796455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:49:39.796455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:49:39.796455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:49:39.796455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:49:39.796455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:49:39.797701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:49:39.797701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:49:39.797701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:49:39.797701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:49:39.797701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:49:39.797701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:49:39.797701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:49:39.797701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:49:39.797701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:49:39.797701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:49:39.797701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:49:39.797701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:49:39.797701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:49:39.797701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:49:39.797701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:49:39.797701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:49:39.797701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:49:39.798658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:49:39.798658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:49:39.798658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:49:39.798658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:49:39.798658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:49:39.798658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:49:39.798658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:49:39.798658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:49:39.798658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 617/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:49:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:49:39.804334 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba8c40-1824-4c1c-9687-7b78923d7c0c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:39.934292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba8c40-1824-4c1c-9687-7b78923d7c0c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:39.960868 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:39.976761 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda68c79-b20f-47f3-80df-7fbbc364ff79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:39.977622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda68c79-b20f-47f3-80df-7fbbc364ff79 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:39.978023 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda68c79-b20f-47f3-80df-7fbbc364ff79 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:39.978458 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda68c79-b20f-47f3-80df-7fbbc364ff79 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:39.979494 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda68c79-b20f-47f3-80df-7fbbc364ff79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:39.979960 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda68c79-b20f-47f3-80df-7fbbc364ff79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:39.981810 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda68c79-b20f-47f3-80df-7fbbc364ff79 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:40.003278 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 601/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:49:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:49:40.005121 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590c49d-fcd7-4f70-90ba-53f6376b98d5 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:40.126744 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590c49d-fcd7-4f70-90ba-53f6376b98d5 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:40.157619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:40.164650 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a039f0-6154-402f-a8b6-0b0fdf76d342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:40.165413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a039f0-6154-402f-a8b6-0b0fdf76d342 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:40.165795 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a039f0-6154-402f-a8b6-0b0fdf76d342 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:40.166162 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a039f0-6154-402f-a8b6-0b0fdf76d342 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:40.167057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a039f0-6154-402f-a8b6-0b0fdf76d342 None admin] RBAC: Adding query filter params (role.id=e5048f4e7c6b4494a15d628758090380, scope.project.id=d583e0be8ef74766bffd6c3c07af18f1, user.id=5049dba7ffde47b8af3e6fd55b5703e2) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:40.167466 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a039f0-6154-402f-a8b6-0b0fdf76d342 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:40.169285 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a039f0-6154-402f-a8b6-0b0fdf76d342 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:40.184078 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 618/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:49:40 2022] GET /identity/v3/role_assignments?user.id=5049dba7ffde47b8af3e6fd55b5703e2&scope.project.id=d583e0be8ef74766bffd6c3c07af18f1&role.id=e5048f4e7c6b4494a15d628758090380 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:49:42.198357 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85815275-2dfa-4a19-b8f2-f6f1891a9f2a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:42.198929 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85815275-2dfa-4a19-b8f2-f6f1891a9f2a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:42.199114 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85815275-2dfa-4a19-b8f2-f6f1891a9f2a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:42.200579 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 602/1220] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:49:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:49:42.206918 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943018e5-1f5b-4e34-bfec-d4a96654717c None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:42.207345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943018e5-1f5b-4e34-bfec-d4a96654717c None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:42.207591 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943018e5-1f5b-4e34-bfec-d4a96654717c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:42.229040 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-943018e5-1f5b-4e34-bfec-d4a96654717c None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:49:42.230364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-943018e5-1f5b-4e34-bfec-d4a96654717c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:49:42.319468 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 619/1221] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:49:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:49:42.345339 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85815275-2dfa-4a19-b8f2-f6f1891a9f2a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:42.464754 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85815275-2dfa-4a19-b8f2-f6f1891a9f2a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:42.506347 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9170f-c93a-4e02-8cd2-2a694693b90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:42.507366 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9170f-c93a-4e02-8cd2-2a694693b90b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:42.507870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9170f-c93a-4e02-8cd2-2a694693b90b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:42.508294 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9170f-c93a-4e02-8cd2-2a694693b90b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:42.520248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c9170f-c93a-4e02-8cd2-2a694693b90b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:49:42.525750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-98c9170f-c93a-4e02-8cd2-2a694693b90b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:49:42.525750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:49:42.525750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:49:42.525750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:49:42.525750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:49:42.525750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:49:42.525750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:49:42.525750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:49:42.525750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:49:42.525750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:49:42.525750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:49:42.525750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:49:42.525750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:49:42.525750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:49:42.525750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:49:42.525750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:49:42.527638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:49:42.527638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:49:42.527638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:49:42.527638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:49:42.527638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:49:42.527638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:49:42.527638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:49:42.527638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:49:42.527638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:49:42.527638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:49:42.527638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:49:42.527638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:49:42.527638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:49:42.527638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:49:42.527638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:49:42.527638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:49:42.529292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:49:42.529292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:49:42.529292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:49:42.529292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:49:42.529292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:49:42.529292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:49:42.529292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:49:42.529292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:49:42.529292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:49:42.529292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:49:42.529292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:49:42.529292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:49:42.529292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 603/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jul 26 04:49:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:49:42.535401 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943018e5-1f5b-4e34-bfec-d4a96654717c None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:42.649422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943018e5-1f5b-4e34-bfec-d4a96654717c None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:42.676411 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:42.689085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a0837-8176-4dc6-a71f-2d614e451918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:42.690000 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a0837-8176-4dc6-a71f-2d614e451918 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:42.690336 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a0837-8176-4dc6-a71f-2d614e451918 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:42.690666 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a0837-8176-4dc6-a71f-2d614e451918 None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:42.691455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a0837-8176-4dc6-a71f-2d614e451918 None admin] RBAC: Adding query filter params (name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:42.691793 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a0837-8176-4dc6-a71f-2d614e451918 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:42.693363 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a0837-8176-4dc6-a71f-2d614e451918 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:42.702664 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 620/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jul 26 04:49:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:49:42.712913 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9170f-c93a-4e02-8cd2-2a694693b90b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:42.758902 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:42.823354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9170f-c93a-4e02-8cd2-2a694693b90b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:42.861766 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43661be3-4709-4fd2-a97b-22a5c3a797ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:42.862441 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43661be3-4709-4fd2-a97b-22a5c3a797ba None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:42.862732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43661be3-4709-4fd2-a97b-22a5c3a797ba None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:42.863014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43661be3-4709-4fd2-a97b-22a5c3a797ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:42.874137 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43661be3-4709-4fd2-a97b-22a5c3a797ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:42.876069 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43661be3-4709-4fd2-a97b-22a5c3a797ba None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:42.892816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-43661be3-4709-4fd2-a97b-22a5c3a797ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:49:42.892816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:49:42.892816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:49:42.892816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:49:42.892816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:49:42.892816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:49:42.892816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:49:42.892816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:49:42.892816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:49:42.892816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:49:42.892816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:49:42.892816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:49:42.892816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:49:42.892816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:49:42.892816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:49:42.894028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:49:42.894028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:49:42.894028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:49:42.894028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:49:42.894028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:49:42.894028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:49:42.894028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:49:42.894028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:49:42.894028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:49:42.894028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:49:42.894028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:49:42.894028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:49:42.894028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:49:42.894028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:49:42.894028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:49:42.894028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:49:42.894028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:49:42.895110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:49:42.895110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:49:42.895110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:49:42.895110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:49:42.895110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 604/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:49:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:49:42.900423 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a0837-8176-4dc6-a71f-2d614e451918 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:43.004711 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a0837-8176-4dc6-a71f-2d614e451918 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:43.041219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549ba83-53f7-445b-a8b2-9a9a6de93534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:43.041777 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549ba83-53f7-445b-a8b2-9a9a6de93534 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:43.042172 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549ba83-53f7-445b-a8b2-9a9a6de93534 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:43.042394 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549ba83-53f7-445b-a8b2-9a9a6de93534 None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:43.043197 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549ba83-53f7-445b-a8b2-9a9a6de93534 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:43.043659 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549ba83-53f7-445b-a8b2-9a9a6de93534 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:43.045153 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549ba83-53f7-445b-a8b2-9a9a6de93534 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:43.059574 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 621/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:49:43.065916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43661be3-4709-4fd2-a97b-22a5c3a797ba None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:43.169850 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:43.177219 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43661be3-4709-4fd2-a97b-22a5c3a797ba None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:43.212664 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65e3ff-d88e-477a-909b-3649202e5415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:43.213359 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65e3ff-d88e-477a-909b-3649202e5415 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:43.213590 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65e3ff-d88e-477a-909b-3649202e5415 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:43.213869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65e3ff-d88e-477a-909b-3649202e5415 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:43.221530 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea65e3ff-d88e-477a-909b-3649202e5415 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:43.222998 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea65e3ff-d88e-477a-909b-3649202e5415 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:43.234362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-ea65e3ff-d88e-477a-909b-3649202e5415 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 04:49:43.234362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:49:43.234362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:49:43.234362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:49:43.234362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:49:43.234362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:49:43.234362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:49:43.234362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:49:43.234362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:49:43.234362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:49:43.234362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:49:43.234362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:49:43.234362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:49:43.234362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:49:43.234362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:49:43.234362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:49:43.235672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:49:43.235672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:49:43.235672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:49:43.235672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:49:43.235672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:49:43.235672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:49:43.235672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:49:43.235672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:49:43.235672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:49:43.235672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:49:43.235672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:49:43.235672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:49:43.235672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:49:43.235672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:49:43.235672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:49:43.235672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:49:43.235672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:49:43.235672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:49:43.236941 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:49:43.236941 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:49:43.236941 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:49:43.236941 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:49:43.236941 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:49:43.236941 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:49:43.236941 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:49:43.236941 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:49:43.236941 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 04:49:43.236941 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:49:43.237780 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 605/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:49:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:49:43.241971 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549ba83-53f7-445b-a8b2-9a9a6de93534 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:43.295788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:43.350274 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549ba83-53f7-445b-a8b2-9a9a6de93534 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:43.387260 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf124cb-5ec7-49b9-8a1c-d75dc71ba399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:43.387969 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf124cb-5ec7-49b9-8a1c-d75dc71ba399 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:43.388252 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf124cb-5ec7-49b9-8a1c-d75dc71ba399 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:43.388551 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf124cb-5ec7-49b9-8a1c-d75dc71ba399 None admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:43.389371 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf124cb-5ec7-49b9-8a1c-d75dc71ba399 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:43.390024 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf124cb-5ec7-49b9-8a1c-d75dc71ba399 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:43.391178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf124cb-5ec7-49b9-8a1c-d75dc71ba399 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:43.414194 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 622/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Jul 26 04:49:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:49:43.415499 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65e3ff-d88e-477a-909b-3649202e5415 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:43.519645 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65e3ff-d88e-477a-909b-3649202e5415 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:43.558404 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b37bda-a77f-480b-b9d2-6b4e081ddc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:43.559049 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b37bda-a77f-480b-b9d2-6b4e081ddc8e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:43.559350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b37bda-a77f-480b-b9d2-6b4e081ddc8e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:43.559659 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b37bda-a77f-480b-b9d2-6b4e081ddc8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:43.571035 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b37bda-a77f-480b-b9d2-6b4e081ddc8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:43.572571 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b37bda-a77f-480b-b9d2-6b4e081ddc8e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:43.588736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-f9b37bda-a77f-480b-b9d2-6b4e081ddc8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:49:43.588736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:49:43.588736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:49:43.588736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:49:43.588736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:49:43.588736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:49:43.588736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:49:43.588736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:49:43.588736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:49:43.588736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:49:43.588736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:49:43.588736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:49:43.588736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:49:43.588736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:49:43.588736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:49:43.590068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:49:43.590068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:49:43.590068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:49:43.590068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:49:43.590068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:49:43.590068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:49:43.590068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:49:43.590068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:49:43.590068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:49:43.590068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:49:43.590068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:49:43.590068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:49:43.590068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:49:43.590068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:49:43.590068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:49:43.590068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:49:43.590068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:49:43.591340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:49:43.591340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:49:43.591340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:49:43.591340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:49:43.591340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 606/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:49:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:49:43.596337 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf124cb-5ec7-49b9-8a1c-d75dc71ba399 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:43.719432 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf124cb-5ec7-49b9-8a1c-d75dc71ba399 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:43.720961 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:43.757983 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f90ffc-5aea-4ccd-8be8-2ec16e6db681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:43.758655 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f90ffc-5aea-4ccd-8be8-2ec16e6db681 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:43.758975 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f90ffc-5aea-4ccd-8be8-2ec16e6db681 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:43.759308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f90ffc-5aea-4ccd-8be8-2ec16e6db681 None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:43.760055 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f90ffc-5aea-4ccd-8be8-2ec16e6db681 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:43.760365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f90ffc-5aea-4ccd-8be8-2ec16e6db681 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:43.761862 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f90ffc-5aea-4ccd-8be8-2ec16e6db681 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:43.774794 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 623/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:49:43.780869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b37bda-a77f-480b-b9d2-6b4e081ddc8e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:43.791299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:43.896467 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b37bda-a77f-480b-b9d2-6b4e081ddc8e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:43.936820 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ba050-4d88-44f7-8c0d-8610a36301a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:43.937851 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ba050-4d88-44f7-8c0d-8610a36301a3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:43.938204 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ba050-4d88-44f7-8c0d-8610a36301a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:43.938559 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ba050-4d88-44f7-8c0d-8610a36301a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:43.947322 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452ba050-4d88-44f7-8c0d-8610a36301a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:43.948853 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452ba050-4d88-44f7-8c0d-8610a36301a3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:43.962584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-452ba050-4d88-44f7-8c0d-8610a36301a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:49:43.962584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:49:43.962584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:49:43.962584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:49:43.962584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:49:43.962584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:49:43.962584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:49:43.962584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:49:43.962584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:49:43.962584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:49:43.962584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:49:43.962584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:49:43.962584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:49:43.962584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:49:43.962584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:49:43.963802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:49:43.963802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:49:43.963802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:49:43.963802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:49:43.963802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:49:43.963802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:49:43.963802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:49:43.963802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:49:43.963802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:49:43.963802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:49:43.963802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:49:43.963802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:49:43.963802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:49:43.963802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:49:43.963802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:49:43.963802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:49:43.963802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:49:43.970872 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:49:43.970872 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:49:43.970872 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:49:43.970872 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:49:43.970872 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:49:43.970872 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:49:43.970872 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:49:43.970872 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:49:43.970872 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 607/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:49:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:49:43.971575 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f90ffc-5aea-4ccd-8be8-2ec16e6db681 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:44.096007 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f90ffc-5aea-4ccd-8be8-2ec16e6db681 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:44.137595 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432732ba-d6c2-4ad0-a61b-5ea37aa26064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:44.138230 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432732ba-d6c2-4ad0-a61b-5ea37aa26064 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:44.138572 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432732ba-d6c2-4ad0-a61b-5ea37aa26064 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:44.138799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432732ba-d6c2-4ad0-a61b-5ea37aa26064 None admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:44.139732 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432732ba-d6c2-4ad0-a61b-5ea37aa26064 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:44.140045 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432732ba-d6c2-4ad0-a61b-5ea37aa26064 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:44.141608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432732ba-d6c2-4ad0-a61b-5ea37aa26064 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:44.155060 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 624/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:49:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:49:44.161580 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ba050-4d88-44f7-8c0d-8610a36301a3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:44.244725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:44.288212 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ba050-4d88-44f7-8c0d-8610a36301a3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:44.326880 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf2c01-25e3-4ce4-95f0-538c45f87668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:44.327597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf2c01-25e3-4ce4-95f0-538c45f87668 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:44.327884 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf2c01-25e3-4ce4-95f0-538c45f87668 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:44.328257 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf2c01-25e3-4ce4-95f0-538c45f87668 None admin] PATH_INFO: `/v3/projects/d583e0be8ef74766bffd6c3c07af18f1/users/5049dba7ffde47b8af3e6fd55b5703e2/roles/e5048f4e7c6b4494a15d628758090380` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:44.343772 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf2c01-25e3-4ce4-95f0-538c45f87668 None admin] RBAC: Authorizing `identity:create_grant(project_id=d583e0be8ef74766bffd6c3c07af18f1, user_id=5049dba7ffde47b8af3e6fd55b5703e2, role_id=e5048f4e7c6b4494a15d628758090380)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:44.345268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf2c01-25e3-4ce4-95f0-538c45f87668 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:44.358957 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 608/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jul 26 04:49:44 2022] PUT /identity/v3/projects/d583e0be8ef74766bffd6c3c07af18f1/users/5049dba7ffde47b8af3e6fd55b5703e2/roles/e5048f4e7c6b4494a15d628758090380 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:49:46.340430 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfcced-e8d0-467a-97fd-6c67815cb9a0 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:46.340977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfcced-e8d0-467a-97fd-6c67815cb9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:46.341096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfcced-e8d0-467a-97fd-6c67815cb9a0 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:46.343045 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 625/1233] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:49:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:49:46.349942 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73270ea2-68d7-4d02-93c4-a0712e4fd463 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:46.350590 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73270ea2-68d7-4d02-93c4-a0712e4fd463 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:46.350763 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73270ea2-68d7-4d02-93c4-a0712e4fd463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:46.371974 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-73270ea2-68d7-4d02-93c4-a0712e4fd463 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:49:46.373197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-73270ea2-68d7-4d02-93c4-a0712e4fd463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:49:46.443656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 609/1234] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:49:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:49:46.466622 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dfcced-e8d0-467a-97fd-6c67815cb9a0 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:46.494082 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:46.577286 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dfcced-e8d0-467a-97fd-6c67815cb9a0 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:46.619272 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6a9b3-f439-4b78-b674-d779a8446424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:46.619967 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6a9b3-f439-4b78-b674-d779a8446424 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:46.620248 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6a9b3-f439-4b78-b674-d779a8446424 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:46.620495 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6a9b3-f439-4b78-b674-d779a8446424 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:46.631160 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea6a9b3-f439-4b78-b674-d779a8446424 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:49:46.636689 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-3ea6a9b3-f439-4b78-b674-d779a8446424 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:49:46.636689 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:49:46.636689 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:49:46.636689 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:49:46.636689 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:49:46.636689 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:49:46.636689 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:49:46.636689 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:49:46.636689 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:49:46.636689 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:49:46.636689 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:49:46.636689 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:49:46.636689 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:49:46.636689 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:49:46.636689 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:49:46.636689 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:49:46.637798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:49:46.637798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:49:46.637798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:49:46.637798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:49:46.637798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:49:46.637798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:49:46.637798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:49:46.637798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:49:46.637798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:49:46.637798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:49:46.637798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:49:46.637798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:49:46.637798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:49:46.637798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:49:46.637798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:49:46.637798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:49:46.638784 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:49:46.638784 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:49:46.638784 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:49:46.638784 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:49:46.638784 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:49:46.638784 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:49:46.638784 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:49:46.638784 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:49:46.638784 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:49:46.638784 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:49:46.638784 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 04:49:46.638784 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:49:46.638784 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 626/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jul 26 04:49:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:49:46.644725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73270ea2-68d7-4d02-93c4-a0712e4fd463 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:46.694241 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:46.751252 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73270ea2-68d7-4d02-93c4-a0712e4fd463 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:46.788286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33801ac1-4371-420e-8079-c54f7dc2103c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:46.789009 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33801ac1-4371-420e-8079-c54f7dc2103c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:46.789381 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33801ac1-4371-420e-8079-c54f7dc2103c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:46.789761 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33801ac1-4371-420e-8079-c54f7dc2103c None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:46.790684 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33801ac1-4371-420e-8079-c54f7dc2103c None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:46.791039 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33801ac1-4371-420e-8079-c54f7dc2103c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:46.792667 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33801ac1-4371-420e-8079-c54f7dc2103c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:46.802828 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 610/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jul 26 04:49:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:49:46.810130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6a9b3-f439-4b78-b674-d779a8446424 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:46.906805 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:46.924278 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6a9b3-f439-4b78-b674-d779a8446424 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:46.965731 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e6da7-7477-45cb-92d0-84b2e808b0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:46.966485 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e6da7-7477-45cb-92d0-84b2e808b0e0 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:46.966839 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e6da7-7477-45cb-92d0-84b2e808b0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:46.967226 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e6da7-7477-45cb-92d0-84b2e808b0e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:46.982068 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e6da7-7477-45cb-92d0-84b2e808b0e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:46.983581 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e6da7-7477-45cb-92d0-84b2e808b0e0 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:47.001036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-d67e6da7-7477-45cb-92d0-84b2e808b0e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:49:47.001036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:49:47.001036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:49:47.001036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:49:47.001036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:49:47.001036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:49:47.001036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:49:47.001036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:49:47.001036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:49:47.001036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:49:47.001036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:49:47.001036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:49:47.001036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:49:47.001036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:49:47.001036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:49:47.002185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:49:47.002185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:49:47.002185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:49:47.002185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:49:47.002185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:49:47.002185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:49:47.002185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:49:47.002185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:49:47.002185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:49:47.002185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:49:47.002185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:49:47.002185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:49:47.002185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:49:47.002185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:49:47.002185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:49:47.002185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:49:47.002185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:49:47.003259 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:49:47.003259 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:49:47.003259 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:49:47.003259 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:49:47.003259 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 627/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:49:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:49:47.011052 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33801ac1-4371-420e-8079-c54f7dc2103c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:47.124084 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33801ac1-4371-420e-8079-c54f7dc2103c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:47.133377 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:47.160208 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7a261-55b1-4b34-93e4-ea052ec7bd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:47.160771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7a261-55b1-4b34-93e4-ea052ec7bd02 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:47.161058 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7a261-55b1-4b34-93e4-ea052ec7bd02 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:47.161335 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7a261-55b1-4b34-93e4-ea052ec7bd02 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:47.162032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7a261-55b1-4b34-93e4-ea052ec7bd02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:47.162336 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7a261-55b1-4b34-93e4-ea052ec7bd02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:47.164520 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7a261-55b1-4b34-93e4-ea052ec7bd02 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:47.177370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 611/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:49:47.183317 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e6da7-7477-45cb-92d0-84b2e808b0e0 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:47.291797 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e6da7-7477-45cb-92d0-84b2e808b0e0 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:47.331509 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a04836-ad63-4845-8301-275e0dcc5203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:47.332279 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a04836-ad63-4845-8301-275e0dcc5203 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:47.332574 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a04836-ad63-4845-8301-275e0dcc5203 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:47.332849 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a04836-ad63-4845-8301-275e0dcc5203 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:47.337489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:47.342300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a04836-ad63-4845-8301-275e0dcc5203 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:47.344097 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a04836-ad63-4845-8301-275e0dcc5203 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:47.357625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-a6a04836-ad63-4845-8301-275e0dcc5203 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 04:49:47.357625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:49:47.357625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:49:47.357625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:49:47.357625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:49:47.357625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:49:47.357625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:49:47.357625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:49:47.357625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:49:47.357625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:49:47.357625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:49:47.357625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:49:47.357625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:49:47.357625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:49:47.357625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:49:47.357625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:49:47.358937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:49:47.358937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:49:47.358937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:49:47.358937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:49:47.358937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:49:47.358937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:49:47.358937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:49:47.358937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:49:47.358937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:49:47.358937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:49:47.358937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:49:47.358937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:49:47.358937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:49:47.358937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:49:47.358937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:49:47.358937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:49:47.358937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:49:47.358937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:49:47.360101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:49:47.360101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:49:47.360101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:49:47.360101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:49:47.360101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:49:47.360101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:49:47.360101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:49:47.360101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:49:47.360101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 04:49:47.360101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:49:47.360101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 628/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:49:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:49:47.364980 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7a261-55b1-4b34-93e4-ea052ec7bd02 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:47.474228 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7a261-55b1-4b34-93e4-ea052ec7bd02 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:47.509799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ebc825-72b8-4f33-9700-45bafd2e1564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:47.510467 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebc825-72b8-4f33-9700-45bafd2e1564 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:47.510750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebc825-72b8-4f33-9700-45bafd2e1564 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:47.511078 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebc825-72b8-4f33-9700-45bafd2e1564 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:47.511881 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ebc825-72b8-4f33-9700-45bafd2e1564 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:47.512193 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ebc825-72b8-4f33-9700-45bafd2e1564 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:47.513674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ebc825-72b8-4f33-9700-45bafd2e1564 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:47.533610 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 612/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Jul 26 04:49:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:49:47.540118 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a04836-ad63-4845-8301-275e0dcc5203 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:47.644259 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a04836-ad63-4845-8301-275e0dcc5203 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:47.684928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa44f4-2f71-4fa9-9e34-eacd98dfad5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:47.685582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa44f4-2f71-4fa9-9e34-eacd98dfad5e None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:47.685866 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa44f4-2f71-4fa9-9e34-eacd98dfad5e None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:47.686147 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa44f4-2f71-4fa9-9e34-eacd98dfad5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:47.760601 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:47.768460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baa44f4-2f71-4fa9-9e34-eacd98dfad5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:47.770238 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baa44f4-2f71-4fa9-9e34-eacd98dfad5e None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:47.884791 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 629/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:49:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 04:49:47.891787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ebc825-72b8-4f33-9700-45bafd2e1564 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:47.922687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:48.001921 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ebc825-72b8-4f33-9700-45bafd2e1564 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:48.051910 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af78ce9-ea51-4ee9-8c0d-5a696985205e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:48.052469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af78ce9-ea51-4ee9-8c0d-5a696985205e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:48.052764 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af78ce9-ea51-4ee9-8c0d-5a696985205e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:48.053116 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af78ce9-ea51-4ee9-8c0d-5a696985205e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:48.064173 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af78ce9-ea51-4ee9-8c0d-5a696985205e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:48.065697 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af78ce9-ea51-4ee9-8c0d-5a696985205e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:48.082191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-2af78ce9-ea51-4ee9-8c0d-5a696985205e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:49:48.082191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:49:48.082191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:49:48.082191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:49:48.082191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:49:48.082191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:49:48.082191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:49:48.082191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:49:48.082191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:49:48.082191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:49:48.082191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:49:48.082191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:49:48.082191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:49:48.082191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:49:48.082191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:49:48.083299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:49:48.083299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:49:48.083299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:49:48.083299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:49:48.083299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:49:48.083299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:49:48.083299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:49:48.083299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:49:48.083299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:49:48.083299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:49:48.083299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:49:48.083299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:49:48.083299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:49:48.083299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:49:48.083299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:49:48.083299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:49:48.083299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:49:48.084249 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:49:48.084249 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:49:48.084249 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:49:48.084249 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:49:48.084249 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 613/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:49:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:49:48.089788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa44f4-2f71-4fa9-9e34-eacd98dfad5e None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:48.197310 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:48.204575 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa44f4-2f71-4fa9-9e34-eacd98dfad5e None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:48.245991 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ec54b-2297-42b4-ba25-e519df7be07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:48.246629 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ec54b-2297-42b4-ba25-e519df7be07e None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:48.246912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ec54b-2297-42b4-ba25-e519df7be07e None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:48.247184 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ec54b-2297-42b4-ba25-e519df7be07e None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:48.247913 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ec54b-2297-42b4-ba25-e519df7be07e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:48.248311 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ec54b-2297-42b4-ba25-e519df7be07e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:48.250044 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ec54b-2297-42b4-ba25-e519df7be07e None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:48.264149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 630/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:49:48.272249 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af78ce9-ea51-4ee9-8c0d-5a696985205e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:48.368244 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:48.392514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af78ce9-ea51-4ee9-8c0d-5a696985205e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:48.431635 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a472d-730f-4a29-b9bf-99b56bc163bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:48.432302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a472d-730f-4a29-b9bf-99b56bc163bd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:48.432589 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a472d-730f-4a29-b9bf-99b56bc163bd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:48.432866 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a472d-730f-4a29-b9bf-99b56bc163bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:48.439736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a472d-730f-4a29-b9bf-99b56bc163bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:48.441196 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a472d-730f-4a29-b9bf-99b56bc163bd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:48.452304 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-271a472d-730f-4a29-b9bf-99b56bc163bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:49:48.452304 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:49:48.452304 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:49:48.452304 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:49:48.452304 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:49:48.452304 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:49:48.452304 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:49:48.452304 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:49:48.452304 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:49:48.452304 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:49:48.452304 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:49:48.452304 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:49:48.452304 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:49:48.452304 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:49:48.452304 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:49:48.453485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:49:48.453485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:49:48.453485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:49:48.453485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:49:48.453485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:49:48.453485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:49:48.453485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:49:48.453485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:49:48.453485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:49:48.453485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:49:48.453485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:49:48.453485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:49:48.453485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:49:48.453485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:49:48.453485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:49:48.453485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:49:48.453485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:49:48.454640 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:49:48.454640 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:49:48.454640 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:49:48.454640 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:49:48.454640 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:49:48.454640 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:49:48.454640 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:49:48.454640 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:49:48.454640 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 614/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:49:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:49:48.462851 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ec54b-2297-42b4-ba25-e519df7be07e None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:48.577163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ec54b-2297-42b4-ba25-e519df7be07e None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:48.613815 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77d301-8377-4b42-ab39-1c610db5d947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:48.614486 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77d301-8377-4b42-ab39-1c610db5d947 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:48.614803 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77d301-8377-4b42-ab39-1c610db5d947 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:48.615101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77d301-8377-4b42-ab39-1c610db5d947 None admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:48.615895 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77d301-8377-4b42-ab39-1c610db5d947 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:48.616230 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77d301-8377-4b42-ab39-1c610db5d947 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:48.617788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77d301-8377-4b42-ab39-1c610db5d947 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:48.631221 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 631/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:49:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:49:48.638006 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a472d-730f-4a29-b9bf-99b56bc163bd None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:48.759501 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a472d-730f-4a29-b9bf-99b56bc163bd None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:48.797287 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e211c7-2384-440f-9492-c316602529aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:48.797976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e211c7-2384-440f-9492-c316602529aa None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:48.798267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e211c7-2384-440f-9492-c316602529aa None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:48.798549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e211c7-2384-440f-9492-c316602529aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:48.799351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e211c7-2384-440f-9492-c316602529aa None admin] RBAC: Adding query filter params (role.id=e5048f4e7c6b4494a15d628758090380, scope.project.id=d583e0be8ef74766bffd6c3c07af18f1, user.id=5049dba7ffde47b8af3e6fd55b5703e2) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:48.799700 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e211c7-2384-440f-9492-c316602529aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:48.801150 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e211c7-2384-440f-9492-c316602529aa None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:48.819469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 615/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:49:48 2022] GET /identity/v3/role_assignments?user.id=5049dba7ffde47b8af3e6fd55b5703e2&scope.project.id=d583e0be8ef74766bffd6c3c07af18f1&role.id=e5048f4e7c6b4494a15d628758090380 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:49:51.094015 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe3421-9902-4544-907a-72ccd9383510 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:51.095731 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe3421-9902-4544-907a-72ccd9383510 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:51.095731 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe3421-9902-4544-907a-72ccd9383510 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:51.096219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 632/1247] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:49:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:49:51.103403 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e787834-1ee9-4f86-bb7d-405968940424 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:51.103967 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e787834-1ee9-4f86-bb7d-405968940424 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:51.104064 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e787834-1ee9-4f86-bb7d-405968940424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:51.111521 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:51.125403 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0e787834-1ee9-4f86-bb7d-405968940424 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:49:51.126652 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0e787834-1ee9-4f86-bb7d-405968940424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:49:51.205182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 616/1248] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:49:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:49:51.231564 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe3421-9902-4544-907a-72ccd9383510 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:51.258642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:51.356476 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe3421-9902-4544-907a-72ccd9383510 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:51.404217 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238aec8-98d9-4076-a09c-6f420a6e928c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:51.405121 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238aec8-98d9-4076-a09c-6f420a6e928c None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:51.405561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238aec8-98d9-4076-a09c-6f420a6e928c None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:51.406046 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238aec8-98d9-4076-a09c-6f420a6e928c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:51.417417 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6238aec8-98d9-4076-a09c-6f420a6e928c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:49:51.422449 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-6238aec8-98d9-4076-a09c-6f420a6e928c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:49:51.422449 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:49:51.422449 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:49:51.422449 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:49:51.422449 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:49:51.422449 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:49:51.422449 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:49:51.422449 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:49:51.422449 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:49:51.422449 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:49:51.422449 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:49:51.422449 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:49:51.422449 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:49:51.422449 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:49:51.422449 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:49:51.422449 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:49:51.423663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:49:51.423663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:49:51.423663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:49:51.423663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:49:51.423663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:49:51.423663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:49:51.423663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:49:51.423663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:49:51.423663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:49:51.423663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:49:51.423663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:49:51.423663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:49:51.423663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:49:51.423663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:49:51.423663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:49:51.423663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:49:51.424659 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:49:51.424659 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:49:51.424659 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:49:51.424659 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:49:51.424659 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:49:51.424659 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:49:51.424659 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:49:51.424659 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:49:51.424659 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:49:51.424659 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:49:51.424659 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:49:51.424659 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:49:51.424659 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 633/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:49:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:49:51.430318 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e787834-1ee9-4f86-bb7d-405968940424 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:51.538697 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e787834-1ee9-4f86-bb7d-405968940424 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:51.575405 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:51.581821 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a786ae7b-4460-4f11-aa62-e6ac690ddade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:51.582464 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786ae7b-4460-4f11-aa62-e6ac690ddade None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:51.582762 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786ae7b-4460-4f11-aa62-e6ac690ddade None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:51.583021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786ae7b-4460-4f11-aa62-e6ac690ddade None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:51.583821 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a786ae7b-4460-4f11-aa62-e6ac690ddade None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:51.584134 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a786ae7b-4460-4f11-aa62-e6ac690ddade None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:51.585557 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a786ae7b-4460-4f11-aa62-e6ac690ddade None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:51.595133 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 617/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:49:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:49:51.601605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238aec8-98d9-4076-a09c-6f420a6e928c None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:51.692460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:51.718870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238aec8-98d9-4076-a09c-6f420a6e928c None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:51.757364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a993a-c201-4302-a896-c6fafa6ffbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:51.758096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a993a-c201-4302-a896-c6fafa6ffbd0 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:51.758392 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a993a-c201-4302-a896-c6fafa6ffbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:51.758665 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a993a-c201-4302-a896-c6fafa6ffbd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:51.770907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a993a-c201-4302-a896-c6fafa6ffbd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:51.772541 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a993a-c201-4302-a896-c6fafa6ffbd0 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:51.792115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-b03a993a-c201-4302-a896-c6fafa6ffbd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:49:51.792115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:49:51.792115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:49:51.792115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:49:51.792115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:49:51.792115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:49:51.792115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:49:51.792115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:49:51.792115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:49:51.792115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:49:51.792115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:49:51.792115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:49:51.792115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:49:51.792115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:49:51.792115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:49:51.793489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:49:51.793489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:49:51.793489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:49:51.793489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:49:51.793489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:49:51.793489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:49:51.793489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:49:51.793489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:49:51.793489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:49:51.793489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:49:51.793489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:49:51.793489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:49:51.793489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:49:51.793489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:49:51.793489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:49:51.793489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:49:51.793489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:49:51.794615 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:49:51.794615 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:49:51.794615 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:49:51.794615 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:49:51.794615 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 634/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:49:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:49:51.800073 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a786ae7b-4460-4f11-aa62-e6ac690ddade None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:51.919179 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a786ae7b-4460-4f11-aa62-e6ac690ddade None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:51.957053 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de7d70-dc61-4a7c-acd4-932ab3559c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:51.957617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de7d70-dc61-4a7c-acd4-932ab3559c66 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:51.957905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de7d70-dc61-4a7c-acd4-932ab3559c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:51.958169 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de7d70-dc61-4a7c-acd4-932ab3559c66 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:51.958872 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de7d70-dc61-4a7c-acd4-932ab3559c66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:51.959182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de7d70-dc61-4a7c-acd4-932ab3559c66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:51.960598 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de7d70-dc61-4a7c-acd4-932ab3559c66 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:51.973164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 618/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:49:51.979115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a993a-c201-4302-a896-c6fafa6ffbd0 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:52.100437 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a993a-c201-4302-a896-c6fafa6ffbd0 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:52.128817 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:52.143104 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289cd09b-2dca-4c98-ac7d-d96e5c11af74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:52.144654 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289cd09b-2dca-4c98-ac7d-d96e5c11af74 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:52.144946 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289cd09b-2dca-4c98-ac7d-d96e5c11af74 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:52.145282 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289cd09b-2dca-4c98-ac7d-d96e5c11af74 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:52.157202 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289cd09b-2dca-4c98-ac7d-d96e5c11af74 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:52.158821 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289cd09b-2dca-4c98-ac7d-d96e5c11af74 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:52.171331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-289cd09b-2dca-4c98-ac7d-d96e5c11af74 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 04:49:52.171331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:49:52.171331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:49:52.171331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:49:52.171331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:49:52.171331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:49:52.171331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:49:52.171331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:49:52.171331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:49:52.171331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:49:52.171331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:49:52.171331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:49:52.171331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:49:52.171331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:49:52.171331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:49:52.171331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:49:52.172422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:49:52.172422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:49:52.172422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:49:52.172422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:49:52.172422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:49:52.172422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:49:52.172422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:49:52.172422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:49:52.172422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:49:52.172422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:49:52.172422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:49:52.172422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:49:52.172422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:49:52.172422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:49:52.172422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:49:52.172422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:49:52.172422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:49:52.172422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:49:52.173475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:49:52.173475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:49:52.173475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:49:52.173475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:49:52.173475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:49:52.173475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:49:52.173475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:49:52.173475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:49:52.173475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 04:49:52.173475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:49:52.173475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 635/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:49:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:49:52.178520 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de7d70-dc61-4a7c-acd4-932ab3559c66 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:52.241415 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:52.289273 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de7d70-dc61-4a7c-acd4-932ab3559c66 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:52.327220 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77251759-4fa1-454d-bce2-c4a4aef2e6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:52.328007 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77251759-4fa1-454d-bce2-c4a4aef2e6b6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:52.328287 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77251759-4fa1-454d-bce2-c4a4aef2e6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:52.328564 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77251759-4fa1-454d-bce2-c4a4aef2e6b6 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:52.329412 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77251759-4fa1-454d-bce2-c4a4aef2e6b6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:52.329678 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77251759-4fa1-454d-bce2-c4a4aef2e6b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:52.331527 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77251759-4fa1-454d-bce2-c4a4aef2e6b6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:52.351687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 619/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Jul 26 04:49:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:49:52.357425 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289cd09b-2dca-4c98-ac7d-d96e5c11af74 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:52.485836 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289cd09b-2dca-4c98-ac7d-d96e5c11af74 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:52.525251 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2819543-27f0-4ed0-9439-0adecaa293f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:52.525934 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2819543-27f0-4ed0-9439-0adecaa293f3 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:52.526345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2819543-27f0-4ed0-9439-0adecaa293f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:52.526612 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2819543-27f0-4ed0-9439-0adecaa293f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:52.591520 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:52.608635 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2819543-27f0-4ed0-9439-0adecaa293f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:52.610501 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2819543-27f0-4ed0-9439-0adecaa293f3 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:52.732119 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 636/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:49:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 04:49:52.738692 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77251759-4fa1-454d-bce2-c4a4aef2e6b6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:52.855835 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77251759-4fa1-454d-bce2-c4a4aef2e6b6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:52.872403 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:52.899529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62627a96-7f0d-47b0-ae87-1595f68ab706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:52.900219 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62627a96-7f0d-47b0-ae87-1595f68ab706 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:52.901876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62627a96-7f0d-47b0-ae87-1595f68ab706 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:52.902164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62627a96-7f0d-47b0-ae87-1595f68ab706 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:52.913050 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62627a96-7f0d-47b0-ae87-1595f68ab706 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:52.914580 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62627a96-7f0d-47b0-ae87-1595f68ab706 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:52.930860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-62627a96-7f0d-47b0-ae87-1595f68ab706 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:49:52.930860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:49:52.930860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:49:52.930860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:49:52.930860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:49:52.930860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:49:52.930860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:49:52.930860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:49:52.930860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:49:52.930860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:49:52.930860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:49:52.930860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:49:52.930860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:49:52.930860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:49:52.930860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:49:52.932057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:49:52.932057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:49:52.932057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:49:52.932057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:49:52.932057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:49:52.932057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:49:52.932057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:49:52.932057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:49:52.932057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:49:52.932057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:49:52.932057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:49:52.932057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:49:52.932057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:49:52.932057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:49:52.932057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:49:52.932057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:49:52.932057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:49:52.933415 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:49:52.933415 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:49:52.933415 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:49:52.933415 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:49:52.933415 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 620/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:49:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:49:52.938364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2819543-27f0-4ed0-9439-0adecaa293f3 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:53.035350 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:53.051727 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2819543-27f0-4ed0-9439-0adecaa293f3 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:53.093278 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d207b-8169-4e86-8047-979eb815f0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:53.094034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d207b-8169-4e86-8047-979eb815f0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:53.094325 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d207b-8169-4e86-8047-979eb815f0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:53.094720 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d207b-8169-4e86-8047-979eb815f0f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:53.095442 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d207b-8169-4e86-8047-979eb815f0f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:53.095808 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d207b-8169-4e86-8047-979eb815f0f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:53.097364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d207b-8169-4e86-8047-979eb815f0f2 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:53.112192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 637/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:49:53.118726 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62627a96-7f0d-47b0-ae87-1595f68ab706 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:53.233765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62627a96-7f0d-47b0-ae87-1595f68ab706 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:53.277290 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2cbe7d-ca94-4d4a-acc4-8a5e4213f905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:53.278060 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2cbe7d-ca94-4d4a-acc4-8a5e4213f905 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:53.278397 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2cbe7d-ca94-4d4a-acc4-8a5e4213f905 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:53.278736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2cbe7d-ca94-4d4a-acc4-8a5e4213f905 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:53.287116 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2cbe7d-ca94-4d4a-acc4-8a5e4213f905 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:53.288634 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2cbe7d-ca94-4d4a-acc4-8a5e4213f905 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:53.301772 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-3d2cbe7d-ca94-4d4a-acc4-8a5e4213f905 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:49:53.301772 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:49:53.301772 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:49:53.301772 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:49:53.301772 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:49:53.301772 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:49:53.301772 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:49:53.301772 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:49:53.301772 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:49:53.301772 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:49:53.301772 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:49:53.301772 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:49:53.301772 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:49:53.301772 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:49:53.301772 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:49:53.307800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:49:53.307800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:49:53.307800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:49:53.307800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:49:53.307800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:49:53.307800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:49:53.307800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:49:53.307800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:49:53.307800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:49:53.307800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:49:53.307800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:49:53.307800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:49:53.307800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:49:53.307800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:49:53.307800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:49:53.307800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:49:53.307800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:49:53.308968 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:49:53.308968 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:49:53.308968 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:49:53.308968 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:49:53.308968 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:49:53.308968 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:49:53.308968 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:49:53.308968 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:49:53.308968 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 621/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:49:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:49:53.309543 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d207b-8169-4e86-8047-979eb815f0f2 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:53.419878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d207b-8169-4e86-8047-979eb815f0f2 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:53.457184 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ed957-e7e9-4699-828f-9bb5770ebc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:53.457898 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ed957-e7e9-4699-828f-9bb5770ebc6f None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:53.458186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ed957-e7e9-4699-828f-9bb5770ebc6f None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:53.458475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ed957-e7e9-4699-828f-9bb5770ebc6f None admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:53.459407 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862ed957-e7e9-4699-828f-9bb5770ebc6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:53.459804 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862ed957-e7e9-4699-828f-9bb5770ebc6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:53.461364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862ed957-e7e9-4699-828f-9bb5770ebc6f None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:53.475604 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 638/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:49:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:49:53.482235 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2cbe7d-ca94-4d4a-acc4-8a5e4213f905 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:53.510612 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:53.591747 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2cbe7d-ca94-4d4a-acc4-8a5e4213f905 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:53.626880 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b11277-c77c-4bf4-9170-d51348d38954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:53.627513 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b11277-c77c-4bf4-9170-d51348d38954 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:53.627823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b11277-c77c-4bf4-9170-d51348d38954 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:53.628099 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b11277-c77c-4bf4-9170-d51348d38954 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:53.628903 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b11277-c77c-4bf4-9170-d51348d38954 None admin] RBAC: Adding query filter params (role.id=72bf8bd23d2c4b76ad32b7b35d6db412, scope.project.id=d583e0be8ef74766bffd6c3c07af18f1, user.id=5049dba7ffde47b8af3e6fd55b5703e2) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:53.629233 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b11277-c77c-4bf4-9170-d51348d38954 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:53.630659 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b11277-c77c-4bf4-9170-d51348d38954 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:53.636897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 622/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:49:53 2022] GET /identity/v3/role_assignments?user.id=5049dba7ffde47b8af3e6fd55b5703e2&scope.project.id=d583e0be8ef74766bffd6c3c07af18f1&role.id=72bf8bd23d2c4b76ad32b7b35d6db412 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:49:55.682628 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8957a5-7139-4b5a-86c2-6ba8d40d05a0 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:55.683177 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8957a5-7139-4b5a-86c2-6ba8d40d05a0 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:55.683177 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8957a5-7139-4b5a-86c2-6ba8d40d05a0 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:55.684471 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 639/1261] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:49:55.691310 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ddd3b-3c2a-4e0f-b14e-30280ea4f614 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:55.692055 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ddd3b-3c2a-4e0f-b14e-30280ea4f614 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:55.692147 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ddd3b-3c2a-4e0f-b14e-30280ea4f614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:55.720231 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0e5ddd3b-3c2a-4e0f-b14e-30280ea4f614 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:49:55.721470 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0e5ddd3b-3c2a-4e0f-b14e-30280ea4f614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:49:55.794296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 623/1262] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:49:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:49:55.823357 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8957a5-7139-4b5a-86c2-6ba8d40d05a0 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:55.835222 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:55.932025 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8957a5-7139-4b5a-86c2-6ba8d40d05a0 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:55.966924 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7eb4c-6e90-4ee6-9ca9-3f0a78cde597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:55.967484 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7eb4c-6e90-4ee6-9ca9-3f0a78cde597 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:55.967794 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7eb4c-6e90-4ee6-9ca9-3f0a78cde597 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:55.968060 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7eb4c-6e90-4ee6-9ca9-3f0a78cde597 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:55.978916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7eb4c-6e90-4ee6-9ca9-3f0a78cde597 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:49:55.983708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-31c7eb4c-6e90-4ee6-9ca9-3f0a78cde597 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:49:55.983708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:49:55.983708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:49:55.983708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:49:55.983708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:49:55.983708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:49:55.983708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:49:55.983708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:49:55.983708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:49:55.983708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:49:55.983708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:49:55.983708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:49:55.983708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:49:55.983708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:49:55.983708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:49:55.983708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:49:55.984940 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:49:55.984940 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:49:55.984940 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:49:55.984940 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:49:55.984940 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:49:55.984940 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:49:55.984940 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:49:55.984940 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:49:55.984940 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:49:55.984940 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:49:55.984940 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:49:55.984940 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:49:55.984940 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:49:55.984940 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:49:55.984940 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:49:55.984940 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:49:55.985907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:49:55.985907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:49:55.985907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:49:55.985907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:49:55.985907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:49:55.985907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:49:55.985907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:49:55.985907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:49:55.985907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:49:55.985907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:49:55.985907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:49:55.985907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:49:55.985907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 640/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:49:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:49:55.991574 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ddd3b-3c2a-4e0f-b14e-30280ea4f614 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:56.019625 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:56.125117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ddd3b-3c2a-4e0f-b14e-30280ea4f614 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:56.163574 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd5166-f094-46e7-a77a-366f90a1917e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:56.164215 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd5166-f094-46e7-a77a-366f90a1917e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:56.164498 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd5166-f094-46e7-a77a-366f90a1917e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:56.164768 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd5166-f094-46e7-a77a-366f90a1917e None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:56.165479 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cd5166-f094-46e7-a77a-366f90a1917e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:56.165786 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cd5166-f094-46e7-a77a-366f90a1917e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:56.167146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cd5166-f094-46e7-a77a-366f90a1917e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:56.176068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 624/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:49:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:49:56.185346 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7eb4c-6e90-4ee6-9ca9-3f0a78cde597 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:56.264437 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:56.301414 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7eb4c-6e90-4ee6-9ca9-3f0a78cde597 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:56.337697 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d3758-c6a4-4d52-9db1-dc1804750fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:56.338339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d3758-c6a4-4d52-9db1-dc1804750fde None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:56.338625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d3758-c6a4-4d52-9db1-dc1804750fde None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:56.338898 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d3758-c6a4-4d52-9db1-dc1804750fde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:56.349574 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d3758-c6a4-4d52-9db1-dc1804750fde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:56.351040 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d3758-c6a4-4d52-9db1-dc1804750fde None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:56.365841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-f22d3758-c6a4-4d52-9db1-dc1804750fde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:49:56.365841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:49:56.365841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:49:56.365841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:49:56.365841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:49:56.365841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:49:56.365841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:49:56.365841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:49:56.365841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:49:56.365841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:49:56.365841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:49:56.365841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:49:56.365841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:49:56.365841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:49:56.365841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:49:56.366881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:49:56.366881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:49:56.366881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:49:56.366881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:49:56.366881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:49:56.366881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:49:56.366881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:49:56.366881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:49:56.366881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:49:56.366881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:49:56.366881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:49:56.366881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:49:56.366881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:49:56.366881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:49:56.366881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:49:56.366881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:49:56.366881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:49:56.367964 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:49:56.367964 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:49:56.367964 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:49:56.367964 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:49:56.367964 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 641/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:49:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:49:56.373004 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd5166-f094-46e7-a77a-366f90a1917e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:56.461693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:56.482313 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd5166-f094-46e7-a77a-366f90a1917e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:56.522702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c1987f-ec10-4394-af2a-24d038002e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:56.523449 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1987f-ec10-4394-af2a-24d038002e80 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:56.523919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1987f-ec10-4394-af2a-24d038002e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:56.524285 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1987f-ec10-4394-af2a-24d038002e80 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:56.525343 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c1987f-ec10-4394-af2a-24d038002e80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:56.525688 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c1987f-ec10-4394-af2a-24d038002e80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:56.527923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c1987f-ec10-4394-af2a-24d038002e80 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:56.542140 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 625/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:49:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:49:56.550275 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d3758-c6a4-4d52-9db1-dc1804750fde None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:56.657024 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d3758-c6a4-4d52-9db1-dc1804750fde None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:56.676710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:56.693171 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf452e2-2a8a-4e51-843a-a70f2a973968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:56.693830 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf452e2-2a8a-4e51-843a-a70f2a973968 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:56.694112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf452e2-2a8a-4e51-843a-a70f2a973968 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:56.694385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf452e2-2a8a-4e51-843a-a70f2a973968 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:56.703396 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf452e2-2a8a-4e51-843a-a70f2a973968 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:56.704934 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf452e2-2a8a-4e51-843a-a70f2a973968 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:56.717065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-0cf452e2-2a8a-4e51-843a-a70f2a973968 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 04:49:56.717065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:49:56.717065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:49:56.717065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:49:56.717065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:49:56.717065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:49:56.717065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:49:56.717065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:49:56.717065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:49:56.717065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:49:56.717065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:49:56.717065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:49:56.717065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:49:56.717065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:49:56.717065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:49:56.717065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:49:56.718089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:49:56.718089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:49:56.718089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:49:56.718089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:49:56.718089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:49:56.718089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:49:56.718089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:49:56.718089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:49:56.718089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:49:56.718089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:49:56.718089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:49:56.718089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:49:56.718089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:49:56.718089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:49:56.718089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:49:56.718089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:49:56.718089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:49:56.718089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:49:56.719244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:49:56.719244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:49:56.719244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:49:56.719244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:49:56.719244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:49:56.719244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:49:56.719244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:49:56.719244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:49:56.719244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 04:49:56.719244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:49:56.719244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 642/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:49:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:49:56.726183 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c1987f-ec10-4394-af2a-24d038002e80 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:56.838147 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c1987f-ec10-4394-af2a-24d038002e80 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:56.882131 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaba56a5-efbf-4b7d-a932-429b05c2c6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:56.882755 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba56a5-efbf-4b7d-a932-429b05c2c6a8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:56.883040 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba56a5-efbf-4b7d-a932-429b05c2c6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:56.883318 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba56a5-efbf-4b7d-a932-429b05c2c6a8 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:56.884155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaba56a5-efbf-4b7d-a932-429b05c2c6a8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:56.884467 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaba56a5-efbf-4b7d-a932-429b05c2c6a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:56.885961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaba56a5-efbf-4b7d-a932-429b05c2c6a8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:56.894908 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:56.908273 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 626/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Jul 26 04:49:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:49:56.916448 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf452e2-2a8a-4e51-843a-a70f2a973968 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:57.024662 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf452e2-2a8a-4e51-843a-a70f2a973968 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:57.064318 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc15305-a70a-4807-ab9b-65328fc44a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:57.065015 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc15305-a70a-4807-ab9b-65328fc44a83 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:57.065297 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc15305-a70a-4807-ab9b-65328fc44a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:57.065646 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc15305-a70a-4807-ab9b-65328fc44a83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:57.077730 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc15305-a70a-4807-ab9b-65328fc44a83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:57.079213 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc15305-a70a-4807-ab9b-65328fc44a83 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:57.094085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-7bc15305-a70a-4807-ab9b-65328fc44a83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:49:57.094085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:49:57.094085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:49:57.094085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:49:57.094085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:49:57.094085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:49:57.094085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:49:57.094085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:49:57.094085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:49:57.094085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:49:57.094085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:49:57.094085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:49:57.094085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:49:57.094085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:49:57.094085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:49:57.095110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:49:57.095110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:49:57.095110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:49:57.095110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:49:57.095110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:49:57.095110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:49:57.095110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:49:57.095110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:49:57.095110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:49:57.095110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:49:57.095110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:49:57.095110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:49:57.095110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:49:57.095110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:49:57.095110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:49:57.095110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:49:57.095110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:49:57.096241 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:49:57.096241 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:49:57.096241 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:49:57.096241 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:49:57.099949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 643/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:49:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:49:57.101696 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaba56a5-efbf-4b7d-a932-429b05c2c6a8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:57.214163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaba56a5-efbf-4b7d-a932-429b05c2c6a8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:57.251675 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0824b7-8d4d-4491-a7c3-7b048da876c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:57.252280 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0824b7-8d4d-4491-a7c3-7b048da876c7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:57.252594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0824b7-8d4d-4491-a7c3-7b048da876c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:57.252872 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0824b7-8d4d-4491-a7c3-7b048da876c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:57.253577 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0824b7-8d4d-4491-a7c3-7b048da876c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:57.253885 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0824b7-8d4d-4491-a7c3-7b048da876c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:57.255273 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0824b7-8d4d-4491-a7c3-7b048da876c7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:57.267669 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 627/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:49:57.273724 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc15305-a70a-4807-ab9b-65328fc44a83 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:57.307076 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:57.383347 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc15305-a70a-4807-ab9b-65328fc44a83 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:57.424328 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f3f25-d325-4c27-a06f-4b3578723189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:57.424980 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f3f25-d325-4c27-a06f-4b3578723189 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:57.425260 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f3f25-d325-4c27-a06f-4b3578723189 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:57.425566 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f3f25-d325-4c27-a06f-4b3578723189 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:57.432356 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22f3f25-d325-4c27-a06f-4b3578723189 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:57.433814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22f3f25-d325-4c27-a06f-4b3578723189 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:57.445163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-d22f3f25-d325-4c27-a06f-4b3578723189 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:49:57.445163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:49:57.445163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:49:57.445163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:49:57.445163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:49:57.445163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:49:57.445163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:49:57.445163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:49:57.445163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:49:57.445163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:49:57.445163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:49:57.445163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:49:57.445163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:49:57.445163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:49:57.445163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:49:57.446224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:49:57.446224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:49:57.446224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:49:57.446224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:49:57.446224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:49:57.446224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:49:57.446224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:49:57.446224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:49:57.446224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:49:57.446224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:49:57.446224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:49:57.446224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:49:57.446224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:49:57.446224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:49:57.446224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:49:57.446224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:49:57.446224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:49:57.447271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:49:57.447271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:49:57.447271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:49:57.447271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:49:57.447271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:49:57.447271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:49:57.447271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:49:57.447271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:49:57.447271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 644/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:49:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 04:49:57.452321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0824b7-8d4d-4491-a7c3-7b048da876c7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:57.518772 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:57.561826 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0824b7-8d4d-4491-a7c3-7b048da876c7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:57.596969 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5763a6f-1b12-4a3d-a472-c0071f3a8d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:57.597580 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5763a6f-1b12-4a3d-a472-c0071f3a8d12 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:57.597866 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5763a6f-1b12-4a3d-a472-c0071f3a8d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:57.598139 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5763a6f-1b12-4a3d-a472-c0071f3a8d12 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:57.598896 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5763a6f-1b12-4a3d-a472-c0071f3a8d12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:49:57.599197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5763a6f-1b12-4a3d-a472-c0071f3a8d12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:57.600677 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5763a6f-1b12-4a3d-a472-c0071f3a8d12 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:57.612629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 628/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:49:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:49:57.619137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f3f25-d325-4c27-a06f-4b3578723189 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:49:57.717346 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:49:57.733128 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f3f25-d325-4c27-a06f-4b3578723189 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:49:57.769818 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6f9e5-6bba-415d-80d8-a409fe2668b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:49:57.770487 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6f9e5-6bba-415d-80d8-a409fe2668b8 None admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:57.770774 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6f9e5-6bba-415d-80d8-a409fe2668b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:57.771052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6f9e5-6bba-415d-80d8-a409fe2668b8 None admin] PATH_INFO: `/v3/projects/d583e0be8ef74766bffd6c3c07af18f1/users/5049dba7ffde47b8af3e6fd55b5703e2/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:57.773308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6f9e5-6bba-415d-80d8-a409fe2668b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=d583e0be8ef74766bffd6c3c07af18f1, user_id=5049dba7ffde47b8af3e6fd55b5703e2, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:49:57.774776 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6f9e5-6bba-415d-80d8-a409fe2668b8 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:49:57.794882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 645/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jul 26 04:49:57 2022] PUT /identity/v3/projects/d583e0be8ef74766bffd6c3c07af18f1/users/5049dba7ffde47b8af3e6fd55b5703e2/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:49:59.780881 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3b4c8-4b1e-4873-bc46-f9e04138d301 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:59.781767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3b4c8-4b1e-4873-bc46-f9e04138d301 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:59.781767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3b4c8-4b1e-4873-bc46-f9e04138d301 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:59.783818 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 629/1274] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:49:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:49:59.792160 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4bb76-4534-456e-b274-ed7040cbec62 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:49:59.792702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4bb76-4534-456e-b274-ed7040cbec62 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:49:59.792842 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4bb76-4534-456e-b274-ed7040cbec62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:49:59.821669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-92d4bb76-4534-456e-b274-ed7040cbec62 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:49:59.822881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-92d4bb76-4534-456e-b274-ed7040cbec62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:49:59.894004 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 646/1275] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:49:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:49:59.917359 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3b4c8-4b1e-4873-bc46-f9e04138d301 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:50:00.021690 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3b4c8-4b1e-4873-bc46-f9e04138d301 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:50:00.040145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:50:00.060477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131b20c-cb5b-439e-8b21-7910d173c01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:50:00.061142 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131b20c-cb5b-439e-8b21-7910d173c01c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:00.061431 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131b20c-cb5b-439e-8b21-7910d173c01c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:00.061710 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131b20c-cb5b-439e-8b21-7910d173c01c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:00.072491 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5131b20c-cb5b-439e-8b21-7910d173c01c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 04:50:00.077562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-5131b20c-cb5b-439e-8b21-7910d173c01c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:50:00.077562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:50:00.077562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:50:00.077562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:50:00.077562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:50:00.077562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:50:00.077562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:50:00.077562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:50:00.077562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:50:00.077562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:50:00.077562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:50:00.077562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:50:00.077562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 04:50:00.077562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 04:50:00.077562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 04:50:00.077562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 04:50:00.078784 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 04:50:00.078784 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 04:50:00.078784 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 04:50:00.078784 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 04:50:00.078784 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:50:00.078784 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:50:00.078784 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:50:00.078784 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:50:00.078784 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:50:00.078784 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:50:00.078784 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:50:00.078784 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:50:00.078784 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:50:00.078784 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:50:00.078784 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:50:00.078784 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:50:00.080102 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:50:00.080102 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:50:00.080102 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:50:00.080102 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:50:00.080102 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 04:50:00.080102 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 04:50:00.080102 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 04:50:00.080102 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 04:50:00.080102 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 04:50:00.080102 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 04:50:00.080102 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 04:50:00.080102 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:50:00.080102 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 630/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jul 26 04:49:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:50:00.085475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4bb76-4534-456e-b274-ed7040cbec62 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:50:00.181350 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:50:00.205479 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4bb76-4534-456e-b274-ed7040cbec62 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:50:00.244385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67e32c-43c9-4298-b98b-4fbfa140aa6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:50:00.245006 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67e32c-43c9-4298-b98b-4fbfa140aa6b None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:00.245289 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67e32c-43c9-4298-b98b-4fbfa140aa6b None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:00.245559 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67e32c-43c9-4298-b98b-4fbfa140aa6b None admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:00.246278 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67e32c-43c9-4298-b98b-4fbfa140aa6b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:50:00.246578 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67e32c-43c9-4298-b98b-4fbfa140aa6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:50:00.247986 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67e32c-43c9-4298-b98b-4fbfa140aa6b None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:50:00.256798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 647/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jul 26 04:50:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:50:00.266090 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131b20c-cb5b-439e-8b21-7910d173c01c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:50:00.378151 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131b20c-cb5b-439e-8b21-7910d173c01c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:50:00.416004 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62ae89-6c85-4fd5-b780-0cec22eb6de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:50:00.416662 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62ae89-6c85-4fd5-b780-0cec22eb6de1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:00.416948 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62ae89-6c85-4fd5-b780-0cec22eb6de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:00.417220 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62ae89-6c85-4fd5-b780-0cec22eb6de1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:00.427944 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62ae89-6c85-4fd5-b780-0cec22eb6de1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:50:00.429402 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62ae89-6c85-4fd5-b780-0cec22eb6de1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:50:00.444429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-3e62ae89-6c85-4fd5-b780-0cec22eb6de1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:50:00.444429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:50:00.444429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:50:00.444429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:50:00.444429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:50:00.444429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:50:00.444429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:50:00.444429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:50:00.444429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:50:00.444429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:50:00.444429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:50:00.444429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:50:00.444429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:50:00.444429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:50:00.444429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:50:00.445469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:50:00.445469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:50:00.445469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:50:00.445469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:50:00.445469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:50:00.445469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:50:00.445469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:50:00.445469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:50:00.445469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:50:00.445469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:50:00.445469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:50:00.445469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:50:00.445469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:50:00.445469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:50:00.445469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:50:00.445469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:50:00.445469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:50:00.446574 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:50:00.446574 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:50:00.446574 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:50:00.446574 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:50:00.446574 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 631/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:50:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:50:00.451603 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67e32c-43c9-4298-b98b-4fbfa140aa6b None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:50:00.576311 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67e32c-43c9-4298-b98b-4fbfa140aa6b None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:50:00.616190 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ab448-5208-4438-9639-0c4d6a34fe31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:50:00.616812 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ab448-5208-4438-9639-0c4d6a34fe31 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:00.617100 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ab448-5208-4438-9639-0c4d6a34fe31 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:00.617394 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ab448-5208-4438-9639-0c4d6a34fe31 None admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:00.618089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ab448-5208-4438-9639-0c4d6a34fe31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:50:00.618400 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ab448-5208-4438-9639-0c4d6a34fe31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:50:00.619819 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ab448-5208-4438-9639-0c4d6a34fe31 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:50:00.632683 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 648/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:50:00.639002 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62ae89-6c85-4fd5-b780-0cec22eb6de1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:50:00.672509 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:50:00.759077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62ae89-6c85-4fd5-b780-0cec22eb6de1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:50:00.796173 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb8485-4ba5-4f60-b5ed-115a59a78270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:50:00.796828 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb8485-4ba5-4f60-b5ed-115a59a78270 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:00.797092 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb8485-4ba5-4f60-b5ed-115a59a78270 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:00.797367 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb8485-4ba5-4f60-b5ed-115a59a78270 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:00.804911 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb8485-4ba5-4f60-b5ed-115a59a78270 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:50:00.806371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb8485-4ba5-4f60-b5ed-115a59a78270 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:50:00.818017 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-bdfb8485-4ba5-4f60-b5ed-115a59a78270 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 04:50:00.818017 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:50:00.818017 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:50:00.818017 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:50:00.818017 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:50:00.818017 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:50:00.818017 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:50:00.818017 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:50:00.818017 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:50:00.818017 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:50:00.818017 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:50:00.818017 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:50:00.818017 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 04:50:00.818017 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 04:50:00.818017 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 04:50:00.818017 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 04:50:00.819095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:50:00.819095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:50:00.819095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 04:50:00.819095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:50:00.819095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 04:50:00.819095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 04:50:00.819095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:50:00.819095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:50:00.819095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:50:00.819095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:50:00.819095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:50:00.819095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:50:00.819095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:50:00.819095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:50:00.819095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:50:00.819095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:50:00.819095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:50:00.819095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:50:00.820199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:50:00.820199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:50:00.820199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 04:50:00.820199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 04:50:00.820199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 04:50:00.820199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 04:50:00.820199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 04:50:00.820199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 04:50:00.820199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 04:50:00.820199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:50:00.820199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 632/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:50:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:50:00.825247 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ab448-5208-4438-9639-0c4d6a34fe31 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:50:00.831272 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:50:00.939114 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ab448-5208-4438-9639-0c4d6a34fe31 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:50:00.975084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263df15-2bec-4cc8-a104-b9eea1230fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:50:00.975748 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263df15-2bec-4cc8-a104-b9eea1230fdd None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:00.976030 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263df15-2bec-4cc8-a104-b9eea1230fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:00.976310 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263df15-2bec-4cc8-a104-b9eea1230fdd None admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:00.977118 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4263df15-2bec-4cc8-a104-b9eea1230fdd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:50:00.977429 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4263df15-2bec-4cc8-a104-b9eea1230fdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:50:00.978892 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4263df15-2bec-4cc8-a104-b9eea1230fdd None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:50:00.997998 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 649/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Jul 26 04:50:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:50:01.004775 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb8485-4ba5-4f60-b5ed-115a59a78270 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:50:01.111183 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:50:01.121953 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb8485-4ba5-4f60-b5ed-115a59a78270 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:50:01.164106 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c766a12-ba20-41c1-89c8-6fd2a5b2cf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:50:01.164739 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c766a12-ba20-41c1-89c8-6fd2a5b2cf38 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:01.165020 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c766a12-ba20-41c1-89c8-6fd2a5b2cf38 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:01.165294 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c766a12-ba20-41c1-89c8-6fd2a5b2cf38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:01.235326 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c766a12-ba20-41c1-89c8-6fd2a5b2cf38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:50:01.236748 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c766a12-ba20-41c1-89c8-6fd2a5b2cf38 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:50:01.331062 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:50:01.345635 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 633/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jul 26 04:50:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 04:50:01.351997 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263df15-2bec-4cc8-a104-b9eea1230fdd None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:50:01.419725 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:50:01.460044 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263df15-2bec-4cc8-a104-b9eea1230fdd None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:50:01.496034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298911d9-28b1-4ea9-96e6-d48dfc6b93af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:50:01.496601 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298911d9-28b1-4ea9-96e6-d48dfc6b93af None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:01.496887 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298911d9-28b1-4ea9-96e6-d48dfc6b93af None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:01.497165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298911d9-28b1-4ea9-96e6-d48dfc6b93af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:01.507765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298911d9-28b1-4ea9-96e6-d48dfc6b93af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:50:01.509257 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298911d9-28b1-4ea9-96e6-d48dfc6b93af None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:50:01.524658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-298911d9-28b1-4ea9-96e6-d48dfc6b93af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:50:01.524658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:50:01.524658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:50:01.524658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:50:01.524658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:50:01.524658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:50:01.524658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:50:01.524658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:50:01.524658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:50:01.524658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:50:01.524658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:50:01.524658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:50:01.524658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 04:50:01.524658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 04:50:01.524658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 04:50:01.525702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 04:50:01.525702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:50:01.525702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:50:01.525702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:50:01.525702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:50:01.525702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:50:01.525702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:50:01.525702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:50:01.525702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:50:01.525702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:50:01.525702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:50:01.525702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:50:01.525702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:50:01.525702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:50:01.525702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:50:01.525702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:50:01.525702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:50:01.526777 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 04:50:01.526777 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 04:50:01.526777 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 04:50:01.526777 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:50:01.526777 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 650/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:50:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:50:01.533792 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c766a12-ba20-41c1-89c8-6fd2a5b2cf38 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:50:01.651675 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c766a12-ba20-41c1-89c8-6fd2a5b2cf38 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:50:01.686872 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b5435-6346-4780-831d-a65fc0dd0c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:50:01.687491 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b5435-6346-4780-831d-a65fc0dd0c43 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:01.687811 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b5435-6346-4780-831d-a65fc0dd0c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:01.688083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b5435-6346-4780-831d-a65fc0dd0c43 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:01.688774 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b5435-6346-4780-831d-a65fc0dd0c43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:50:01.689075 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b5435-6346-4780-831d-a65fc0dd0c43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:50:01.690438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b5435-6346-4780-831d-a65fc0dd0c43 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:50:01.701166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 634/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jul 26 04:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:50:01.707787 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298911d9-28b1-4ea9-96e6-d48dfc6b93af None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:50:01.839087 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298911d9-28b1-4ea9-96e6-d48dfc6b93af None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:50:01.854211 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:50:01.876841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec0821-8a85-4f75-9f0e-e7afe0917c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:50:01.877483 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec0821-8a85-4f75-9f0e-e7afe0917c26 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:01.877768 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec0821-8a85-4f75-9f0e-e7afe0917c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:01.878039 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec0821-8a85-4f75-9f0e-e7afe0917c26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:01.885161 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec0821-8a85-4f75-9f0e-e7afe0917c26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:50:01.886644 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec0821-8a85-4f75-9f0e-e7afe0917c26 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:50:01.898737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application [None req-b1ec0821-8a85-4f75-9f0e-e7afe0917c26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:50:01.898737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:50:01.898737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:50:01.898737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:50:01.898737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:50:01.898737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:50:01.898737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:50:01.898737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:50:01.898737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:50:01.898737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:50:01.898737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:50:01.898737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:50:01.898737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 04:50:01.898737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 04:50:01.898737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 04:50:01.899838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 04:50:01.899838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:50:01.899838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:50:01.899838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:50:01.899838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:50:01.899838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:50:01.899838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:50:01.899838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:50:01.899838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application with Lock( Jul 26 04:50:01.899838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:50:01.899838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:50:01.899838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:50:01.899838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:50:01.899838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:50:01.899838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:50:01.899838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:50:01.899838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:50:01.900923 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 04:50:01.900923 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 04:50:01.900923 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 04:50:01.900923 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 04:50:01.900923 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 04:50:01.900923 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 04:50:01.900923 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 04:50:01.900923 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: ERROR keystone.server.flask.application Jul 26 04:50:01.900923 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 651/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jul 26 04:50:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:50:01.906244 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b5435-6346-4780-831d-a65fc0dd0c43 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:50:01.962035 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:50:02.023106 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b5435-6346-4780-831d-a65fc0dd0c43 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:50:02.064436 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86853dcf-a65c-4f7f-b54a-8323ddf41549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:50:02.065045 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86853dcf-a65c-4f7f-b54a-8323ddf41549 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:02.065535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86853dcf-a65c-4f7f-b54a-8323ddf41549 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:02.065959 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86853dcf-a65c-4f7f-b54a-8323ddf41549 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:02.066689 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86853dcf-a65c-4f7f-b54a-8323ddf41549 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:50:02.067024 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86853dcf-a65c-4f7f-b54a-8323ddf41549 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:50:02.068794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86853dcf-a65c-4f7f-b54a-8323ddf41549 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:50:02.082045 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 635/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jul 26 04:50:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:50:02.088625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec0821-8a85-4f75-9f0e-e7afe0917c26 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:50:02.195928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec0821-8a85-4f75-9f0e-e7afe0917c26 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:50:02.230915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4abf0f2-1f09-49a1-b44f-7275f47aced9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:50:02.231512 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4abf0f2-1f09-49a1-b44f-7275f47aced9 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:02.231831 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4abf0f2-1f09-49a1-b44f-7275f47aced9 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:02.232105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4abf0f2-1f09-49a1-b44f-7275f47aced9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:02.232892 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4abf0f2-1f09-49a1-b44f-7275f47aced9 None admin] RBAC: Adding query filter params (role.id=72bf8bd23d2c4b76ad32b7b35d6db412, scope.project.id=d583e0be8ef74766bffd6c3c07af18f1, user.id=5049dba7ffde47b8af3e6fd55b5703e2) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:50:02.233199 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4abf0f2-1f09-49a1-b44f-7275f47aced9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:50:02.234750 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4abf0f2-1f09-49a1-b44f-7275f47aced9 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:50:02.241662 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 652/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jul 26 04:50:02 2022] GET /identity/v3/role_assignments?user.id=5049dba7ffde47b8af3e6fd55b5703e2&scope.project.id=d583e0be8ef74766bffd6c3c07af18f1&role.id=72bf8bd23d2c4b76ad32b7b35d6db412 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:50:04.275454 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0661d1-314c-4e56-b3a9-7a567d8293de None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:04.276431 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0661d1-314c-4e56-b3a9-7a567d8293de None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:04.276431 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0661d1-314c-4e56-b3a9-7a567d8293de None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:04.278488 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 636/1288] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:50:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:50:04.285200 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77752ea5-7819-4ffb-8395-25e7462d6405 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:04.285873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77752ea5-7819-4ffb-8395-25e7462d6405 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:04.286091 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77752ea5-7819-4ffb-8395-25e7462d6405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:04.309812 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-77752ea5-7819-4ffb-8395-25e7462d6405 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:50:04.311034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-77752ea5-7819-4ffb-8395-25e7462d6405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:50:04.348887 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:50:04.392062 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 653/1289] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:50:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:50:04.425925 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0661d1-314c-4e56-b3a9-7a567d8293de None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:50:04.547378 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0661d1-314c-4e56-b3a9-7a567d8293de None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:50:04.558388 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:50:04.584721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70b3e3-68a7-464f-9d44-91e7db4315e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:50:04.585366 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70b3e3-68a7-464f-9d44-91e7db4315e6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:04.585651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70b3e3-68a7-464f-9d44-91e7db4315e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:04.585925 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70b3e3-68a7-464f-9d44-91e7db4315e6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:04.586558 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70b3e3-68a7-464f-9d44-91e7db4315e6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:50:04.587994 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70b3e3-68a7-464f-9d44-91e7db4315e6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:50:04.597614 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-3c70b3e3-68a7-464f-9d44-91e7db4315e6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 04:50:04.597614 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:50:04.597614 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 04:50:04.597614 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 04:50:04.597614 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 04:50:04.597614 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 04:50:04.597614 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 04:50:04.597614 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:50:04.597614 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 04:50:04.597614 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:50:04.597614 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:50:04.597614 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:50:04.597614 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:50:04.597614 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:50:04.597614 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:50:04.597614 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:50:04.597614 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:50:04.598999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:50:04.598999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:50:04.598999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:50:04.598999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:50:04.598999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:50:04.598999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:50:04.598999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 04:50:04.598999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 04:50:04.598999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 04:50:04.598999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 04:50:04.598999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:50:04.598999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:50:04.598999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:50:04.598999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:50:04.598999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:50:04.598999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:50:04.600163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:50:04.600163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:50:04.600163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:50:04.600163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:50:04.600163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:50:04.600163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:50:04.600163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:50:04.600163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:50:04.600163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:50:04.600163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:50:04.600163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 04:50:04.600163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:50:04.600163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 04:50:04.600163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:50:04.600163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 637/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Jul 26 04:50:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:50:04.606961 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77752ea5-7819-4ffb-8395-25e7462d6405 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:50:04.718158 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77752ea5-7819-4ffb-8395-25e7462d6405 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:50:04.752994 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d220eb-35ee-44c9-9e1e-e70d85cf66ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:50:04.753603 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d220eb-35ee-44c9-9e1e-e70d85cf66ea None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:04.753879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d220eb-35ee-44c9-9e1e-e70d85cf66ea None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:04.754149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d220eb-35ee-44c9-9e1e-e70d85cf66ea None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:04.754826 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d220eb-35ee-44c9-9e1e-e70d85cf66ea None admin] RBAC: Adding query filter params (name=placement) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:50:04.755126 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d220eb-35ee-44c9-9e1e-e70d85cf66ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:50:04.756531 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d220eb-35ee-44c9-9e1e-e70d85cf66ea None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:50:04.761521 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 654/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Tue Jul 26 04:50:04 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:50:04.768523 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70b3e3-68a7-464f-9d44-91e7db4315e6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:50:04.881407 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70b3e3-68a7-464f-9d44-91e7db4315e6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:50:04.916488 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3c66a-37e7-4a72-be73-ff101dc195a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:50:04.917111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3c66a-37e7-4a72-be73-ff101dc195a1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:04.917396 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3c66a-37e7-4a72-be73-ff101dc195a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:04.917673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3c66a-37e7-4a72-be73-ff101dc195a1 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:04.918359 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3c66a-37e7-4a72-be73-ff101dc195a1 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:50:04.918670 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3c66a-37e7-4a72-be73-ff101dc195a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:50:04.920076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3c66a-37e7-4a72-be73-ff101dc195a1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:50:04.925309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 638/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Tue Jul 26 04:50:04 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:50:06.883791 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945df01-f059-4896-9005-115d02c8cf60 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:06.884340 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945df01-f059-4896-9005-115d02c8cf60 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:06.884340 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945df01-f059-4896-9005-115d02c8cf60 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:06.885742 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 655/1293] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:50:06.892154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f1f6b-ed03-427f-9302-a278ce3430f5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:06.892909 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f1f6b-ed03-427f-9302-a278ce3430f5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:06.893053 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f1f6b-ed03-427f-9302-a278ce3430f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:06.917222 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-238f1f6b-ed03-427f-9302-a278ce3430f5 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:50:06.918448 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-238f1f6b-ed03-427f-9302-a278ce3430f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:50:06.950341 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:50:06.991135 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 639/1294] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:50:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:50:07.016618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945df01-f059-4896-9005-115d02c8cf60 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:50:07.053962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:50:07.142854 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945df01-f059-4896-9005-115d02c8cf60 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:50:07.179861 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97214752-acdf-45db-ae24-9398aa7dd405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:50:07.180478 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97214752-acdf-45db-ae24-9398aa7dd405 None admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:07.180763 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97214752-acdf-45db-ae24-9398aa7dd405 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:07.181042 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97214752-acdf-45db-ae24-9398aa7dd405 None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:07.181745 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97214752-acdf-45db-ae24-9398aa7dd405 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:50:07.183135 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97214752-acdf-45db-ae24-9398aa7dd405 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:50:07.199320 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 656/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jul 26 04:50:07 2022] POST /identity/v3/services => generated 246 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:50:09.211931 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936f265-986a-4dfd-ae03-9f70449f24ff None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:09.212861 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936f265-986a-4dfd-ae03-9f70449f24ff None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:09.212861 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936f265-986a-4dfd-ae03-9f70449f24ff None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:09.214976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 640/1296] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:50:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:50:09.222882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166f772-9719-4401-8ae1-3a3322b81e91 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:09.223327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166f772-9719-4401-8ae1-3a3322b81e91 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:09.223418 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166f772-9719-4401-8ae1-3a3322b81e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:09.246373 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-1166f772-9719-4401-8ae1-3a3322b81e91 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:50:09.248359 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-1166f772-9719-4401-8ae1-3a3322b81e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:50:09.337160 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 657/1297] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:50:09 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 04:50:09.359786 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936f265-986a-4dfd-ae03-9f70449f24ff None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:50:09.471352 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936f265-986a-4dfd-ae03-9f70449f24ff None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:50:09.508428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3da829-2b09-45e4-8cf9-dbaa32a6873b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:50:09.509068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3da829-2b09-45e4-8cf9-dbaa32a6873b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:09.509348 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3da829-2b09-45e4-8cf9-dbaa32a6873b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:09.509622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3da829-2b09-45e4-8cf9-dbaa32a6873b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:09.510247 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3da829-2b09-45e4-8cf9-dbaa32a6873b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:50:09.511654 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3da829-2b09-45e4-8cf9-dbaa32a6873b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:50:09.521027 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-7e3da829-2b09-45e4-8cf9-dbaa32a6873b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 04:50:09.521027 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:50:09.521027 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 04:50:09.521027 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 04:50:09.521027 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 04:50:09.521027 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 04:50:09.521027 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 04:50:09.521027 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:50:09.521027 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 04:50:09.521027 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:50:09.521027 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:50:09.521027 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:50:09.521027 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:50:09.521027 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:50:09.521027 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:50:09.521027 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:50:09.521027 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:50:09.522057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:50:09.522057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:50:09.522057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:50:09.522057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:50:09.522057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:50:09.522057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:50:09.522057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 04:50:09.522057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 04:50:09.522057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 04:50:09.522057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 04:50:09.522057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:50:09.522057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:50:09.522057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:50:09.522057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:50:09.522057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:50:09.522057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:50:09.523009 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:50:09.523009 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:50:09.523009 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:50:09.523009 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:50:09.523009 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:50:09.523009 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:50:09.523009 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:50:09.523009 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:50:09.523009 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:50:09.523009 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:50:09.523009 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 04:50:09.523009 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:50:09.523009 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 04:50:09.523009 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:50:09.523009 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 641/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Jul 26 04:50:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:50:09.529619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166f772-9719-4401-8ae1-3a3322b81e91 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:50:09.554453 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:50:09.639822 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166f772-9719-4401-8ae1-3a3322b81e91 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:50:09.674350 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc640c92-c47e-472f-a3a6-27697a8fe734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:50:09.674962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc640c92-c47e-472f-a3a6-27697a8fe734 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:09.675247 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc640c92-c47e-472f-a3a6-27697a8fe734 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:09.675528 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc640c92-c47e-472f-a3a6-27697a8fe734 None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:09.676249 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc640c92-c47e-472f-a3a6-27697a8fe734 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:50:09.676565 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc640c92-c47e-472f-a3a6-27697a8fe734 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:50:09.677952 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc640c92-c47e-472f-a3a6-27697a8fe734 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:50:09.683129 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 658/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Tue Jul 26 04:50:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:50:09.689647 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3da829-2b09-45e4-8cf9-dbaa32a6873b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:50:09.722735 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:50:09.801377 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3da829-2b09-45e4-8cf9-dbaa32a6873b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:50:09.835404 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0fc9cf-a80d-4144-a8a8-16b3467cb9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:50:09.836063 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0fc9cf-a80d-4144-a8a8-16b3467cb9c0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:09.836343 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0fc9cf-a80d-4144-a8a8-16b3467cb9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:09.836619 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0fc9cf-a80d-4144-a8a8-16b3467cb9c0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:09.837367 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0fc9cf-a80d-4144-a8a8-16b3467cb9c0 None admin] RBAC: Adding query filter params (interface=public, service_id=4540f8e41a7b4a4eaa89d36d22350293, region_id=RegionOne) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:50:09.837677 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0fc9cf-a80d-4144-a8a8-16b3467cb9c0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:50:09.839050 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0fc9cf-a80d-4144-a8a8-16b3467cb9c0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:50:09.844310 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 642/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Jul 26 04:50:09 2022] GET /identity/v3/endpoints?service_id=4540f8e41a7b4a4eaa89d36d22350293&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:50:09.850478 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc640c92-c47e-472f-a3a6-27697a8fe734 None admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:50:09.939416 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:50:09.961310 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc640c92-c47e-472f-a3a6-27697a8fe734 None admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:50:09.999715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f11a5-53ca-4d97-b0f5-aeedcd60f4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:50:10.000138 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f11a5-53ca-4d97-b0f5-aeedcd60f4ce None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:10.000418 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f11a5-53ca-4d97-b0f5-aeedcd60f4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:10.000688 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f11a5-53ca-4d97-b0f5-aeedcd60f4ce None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:10.001336 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f11a5-53ca-4d97-b0f5-aeedcd60f4ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:50:10.002785 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f11a5-53ca-4d97-b0f5-aeedcd60f4ce None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:50:10.010169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 659/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Jul 26 04:50:09 2022] GET /identity/v3/services => generated 2191 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 04:50:11.998140 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b4be1-de4f-49b5-96a8-4194618dd859 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:11.998738 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b4be1-de4f-49b5-96a8-4194618dd859 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:11.998738 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b4be1-de4f-49b5-96a8-4194618dd859 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:12.000111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 643/1302] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:50:12.006689 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6fc3f8-fd03-4b17-9e7f-ab43e13b0ea0 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:12.007382 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6fc3f8-fd03-4b17-9e7f-ab43e13b0ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:12.007616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6fc3f8-fd03-4b17-9e7f-ab43e13b0ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:12.035819 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-fc6fc3f8-fd03-4b17-9e7f-ab43e13b0ea0 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:50:12.037149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-fc6fc3f8-fd03-4b17-9e7f-ab43e13b0ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:50:12.115878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 660/1303] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:50:12 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 04:50:12.138890 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b4be1-de4f-49b5-96a8-4194618dd859 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:50:12.242660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:50:12.250623 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b4be1-de4f-49b5-96a8-4194618dd859 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:50:12.288362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72ee2c-4d42-4b43-93ba-323d5bf9f154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:50:12.289012 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72ee2c-4d42-4b43-93ba-323d5bf9f154 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:12.289331 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72ee2c-4d42-4b43-93ba-323d5bf9f154 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:12.289595 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72ee2c-4d42-4b43-93ba-323d5bf9f154 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:12.290224 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72ee2c-4d42-4b43-93ba-323d5bf9f154 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:50:12.291633 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72ee2c-4d42-4b43-93ba-323d5bf9f154 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:50:12.300758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-9c72ee2c-4d42-4b43-93ba-323d5bf9f154 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 04:50:12.300758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:50:12.300758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 04:50:12.300758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 04:50:12.300758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 04:50:12.300758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 04:50:12.300758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 04:50:12.300758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:50:12.300758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 04:50:12.300758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:50:12.300758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 04:50:12.300758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:50:12.300758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 04:50:12.300758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 04:50:12.300758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 04:50:12.300758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 04:50:12.300758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 04:50:12.301862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 04:50:12.301862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 04:50:12.301862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 04:50:12.301862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 04:50:12.301862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 04:50:12.301862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 04:50:12.301862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 04:50:12.301862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 04:50:12.301862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 04:50:12.301862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 04:50:12.301862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 04:50:12.301862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 04:50:12.301862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 04:50:12.301862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 04:50:12.301862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 04:50:12.301862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 04:50:12.302687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 04:50:12.302687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 26 04:50:12.302687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 04:50:12.302687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 26 04:50:12.302687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 04:50:12.302687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 04:50:12.302687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 04:50:12.302687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 26 04:50:12.302687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 04:50:12.302687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 26 04:50:12.302687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 04:50:12.302687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 04:50:12.302687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 04:50:12.302687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 26 04:50:12.302687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 644/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Jul 26 04:50:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 04:50:12.308437 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6fc3f8-fd03-4b17-9e7f-ab43e13b0ea0 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:50:12.388844 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:50:12.412610 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6fc3f8-fd03-4b17-9e7f-ab43e13b0ea0 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:50:12.447315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574409b3-e38c-40a1-86da-ef97e31afe78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:50:12.447959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574409b3-e38c-40a1-86da-ef97e31afe78 None admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:12.448240 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574409b3-e38c-40a1-86da-ef97e31afe78 None admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:12.448511 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574409b3-e38c-40a1-86da-ef97e31afe78 None admin] PATH_INFO: `/v3/services` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:12.449186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574409b3-e38c-40a1-86da-ef97e31afe78 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:50:12.449470 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574409b3-e38c-40a1-86da-ef97e31afe78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:50:12.450876 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574409b3-e38c-40a1-86da-ef97e31afe78 None admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:50:12.456606 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 661/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Tue Jul 26 04:50:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:50:12.465700 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72ee2c-4d42-4b43-93ba-323d5bf9f154 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:50:12.579572 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72ee2c-4d42-4b43-93ba-323d5bf9f154 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:50:12.615546 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0274c-de90-46b9-a151-f722cfa5298a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:50:12.616197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0274c-de90-46b9-a151-f722cfa5298a None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:50:12.616480 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0274c-de90-46b9-a151-f722cfa5298a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:50:12.616762 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0274c-de90-46b9-a151-f722cfa5298a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:50:12.617472 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef0274c-de90-46b9-a151-f722cfa5298a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:50:12.618848 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef0274c-de90-46b9-a151-f722cfa5298a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:50:12.640651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 645/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 04:50:12 2022] POST /identity/v3/endpoints => generated 331 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 04:50:22.453060 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=30 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:50:22.635751 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:51:16.554605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87dc98-e7a4-4fb0-9ca5-9973778e5a93 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:16.555197 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87dc98-e7a4-4fb0-9ca5-9973778e5a93 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:16.555197 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87dc98-e7a4-4fb0-9ca5-9973778e5a93 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:16.556502 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 662/1307] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:51:16.564424 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c7bd9-5312-4d14-923a-a9a035cad2ca None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:16.565087 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c7bd9-5312-4d14-923a-a9a035cad2ca None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:16.565087 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c7bd9-5312-4d14-923a-a9a035cad2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:16.687625 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ce6c7bd9-5312-4d14-923a-a9a035cad2ca None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:51:16.692568 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ce6c7bd9-5312-4d14-923a-a9a035cad2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:16.898449 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 646/1308] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:51:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:51:16.949325 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31dd22-c9b2-4d26-93fb-a8db0294f68d None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:16.950117 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31dd22-c9b2-4d26-93fb-a8db0294f68d None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:16.950582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31dd22-c9b2-4d26-93fb-a8db0294f68d None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:16.952631 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 663/1309] 10.222.0.17 () {54 vars in 965 bytes} [Tue Jul 26 04:51:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:51:16.959287 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391f170-f1e8-4480-90ba-707b95cc4125 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:16.959615 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391f170-f1e8-4480-90ba-707b95cc4125 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:16.959903 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391f170-f1e8-4480-90ba-707b95cc4125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:17.041715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d391f170-f1e8-4480-90ba-707b95cc4125 None None] MFA Rules not processed for user `dbdb8997891241e9b2e829feb8023f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:51:17.042957 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d391f170-f1e8-4480-90ba-707b95cc4125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:17.065959 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:51:17.125174 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 647/1310] 10.222.0.17 () {58 vars in 1062 bytes} [Tue Jul 26 04:51:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 04:51:17.134533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31dd22-c9b2-4d26-93fb-a8db0294f68d None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:51:17.136411 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-bd31dd22-c9b2-4d26-93fb-a8db0294f68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:17.262405 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31dd22-c9b2-4d26-93fb-a8db0294f68d None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:51:17.350480 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357fb97c-8245-4b22-9f57-61d2ca4b9d8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:51:17.351135 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fb97c-8245-4b22-9f57-61d2ca4b9d8a service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:17.351407 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fb97c-8245-4b22-9f57-61d2ca4b9d8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:17.351724 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fb97c-8245-4b22-9f57-61d2ca4b9d8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:17.353410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-357fb97c-8245-4b22-9f57-61d2ca4b9d8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:17.419101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:51:17.480048 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357fb97c-8245-4b22-9f57-61d2ca4b9d8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:51:17.481567 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357fb97c-8245-4b22-9f57-61d2ca4b9d8a service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:51:17.501236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 664/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:51:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:51:19.787211 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99ef5b-b536-4406-b385-82d5749cdc98 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:19.787781 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99ef5b-b536-4406-b385-82d5749cdc98 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:19.787781 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99ef5b-b536-4406-b385-82d5749cdc98 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:19.789128 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 648/1312] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:51:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:51:19.794607 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e20984-b6e4-4c76-9a40-b1dd7ed19541 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:19.795048 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e20984-b6e4-4c76-9a40-b1dd7ed19541 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:19.795128 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e20984-b6e4-4c76-9a40-b1dd7ed19541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:19.871513 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-74e20984-b6e4-4c76-9a40-b1dd7ed19541 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:51:19.872886 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-74e20984-b6e4-4c76-9a40-b1dd7ed19541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:19.983075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 665/1313] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:51:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:51:20.021384 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d99ef5b-b536-4406-b385-82d5749cdc98 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:51:20.023164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2d99ef5b-b536-4406-b385-82d5749cdc98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:20.142213 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d99ef5b-b536-4406-b385-82d5749cdc98 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:51:20.220664 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:51:20.223420 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68987051-1a0a-4a14-a389-ef64cdf8f027 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:51:20.224110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68987051-1a0a-4a14-a389-ef64cdf8f027 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:20.224396 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68987051-1a0a-4a14-a389-ef64cdf8f027 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:20.224673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68987051-1a0a-4a14-a389-ef64cdf8f027 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:20.226982 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-68987051-1a0a-4a14-a389-ef64cdf8f027 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:20.351703 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68987051-1a0a-4a14-a389-ef64cdf8f027 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:51:20.353155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68987051-1a0a-4a14-a389-ef64cdf8f027 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:51:20.372860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 649/1314] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:51:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:51:22.734981 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d856f-05e6-4c98-8049-556191c14820 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:22.735717 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d856f-05e6-4c98-8049-556191c14820 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:22.735717 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d856f-05e6-4c98-8049-556191c14820 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:22.736986 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 666/1315] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:51:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:51:22.742455 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064bbf2-a296-466e-96f1-ace79ff8b5fa None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:22.742866 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064bbf2-a296-466e-96f1-ace79ff8b5fa None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:22.743021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064bbf2-a296-466e-96f1-ace79ff8b5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:22.811711 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4064bbf2-a296-466e-96f1-ace79ff8b5fa None None] MFA Rules not processed for user `7b1c7d019a3b49b79a0a9a3a4aea88f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:51:22.812941 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4064bbf2-a296-466e-96f1-ace79ff8b5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:22.866993 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:51:22.915875 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 650/1316] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:51:22 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:51:22.957194 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d856f-05e6-4c98-8049-556191c14820 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:51:22.959508 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-521d856f-05e6-4c98-8049-556191c14820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:22.974269 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:51:23.077601 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d856f-05e6-4c98-8049-556191c14820 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:51:23.156332 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f237ccca-cd20-4f82-9380-e4c38c803d22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:51:23.156958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237ccca-cd20-4f82-9380-e4c38c803d22 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:23.157231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237ccca-cd20-4f82-9380-e4c38c803d22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:23.157498 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237ccca-cd20-4f82-9380-e4c38c803d22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:23.159131 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-f237ccca-cd20-4f82-9380-e4c38c803d22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:23.225243 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:51:23.280920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f237ccca-cd20-4f82-9380-e4c38c803d22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:51:23.282382 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f237ccca-cd20-4f82-9380-e4c38c803d22 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:51:23.304002 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 667/1317] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:51:22 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:51:25.923012 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef2ca8-20d2-48a0-beb7-2d0a88966ac9 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:25.923642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef2ca8-20d2-48a0-beb7-2d0a88966ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:25.923642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef2ca8-20d2-48a0-beb7-2d0a88966ac9 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:25.925013 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 651/1318] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:51:25.931342 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fbf1e-0070-462d-b1cd-d01088d89f62 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:25.931848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fbf1e-0070-462d-b1cd-d01088d89f62 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:25.931958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fbf1e-0070-462d-b1cd-d01088d89f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:26.014612 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-c69fbf1e-0070-462d-b1cd-d01088d89f62 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:51:26.015897 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c69fbf1e-0070-462d-b1cd-d01088d89f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:26.120146 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 668/1319] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:51:25 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:51:26.177301 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d257b-4957-41eb-8c37-7f74b8103d94 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:26.177527 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d257b-4957-41eb-8c37-7f74b8103d94 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:26.177994 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d257b-4957-41eb-8c37-7f74b8103d94 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:26.179378 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 652/1320] 10.222.0.17 () {54 vars in 965 bytes} [Tue Jul 26 04:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:51:26.187467 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd21729-b247-4635-bf3d-e64a62cb85a0 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:26.187820 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd21729-b247-4635-bf3d-e64a62cb85a0 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:26.188182 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd21729-b247-4635-bf3d-e64a62cb85a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:26.215719 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:51:26.271229 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-5dd21729-b247-4635-bf3d-e64a62cb85a0 None None] MFA Rules not processed for user `dbdb8997891241e9b2e829feb8023f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:51:26.272487 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-5dd21729-b247-4635-bf3d-e64a62cb85a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:26.366738 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 669/1321] 10.222.0.17 () {58 vars in 1062 bytes} [Tue Jul 26 04:51:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:51:26.376441 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d257b-4957-41eb-8c37-7f74b8103d94 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:51:26.378012 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e51d257b-4957-41eb-8c37-7f74b8103d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:26.487700 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d257b-4957-41eb-8c37-7f74b8103d94 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:51:26.580725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788171ba-04a9-4a13-98ce-a9aac4bfb297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:51:26.581356 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788171ba-04a9-4a13-98ce-a9aac4bfb297 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:26.581635 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788171ba-04a9-4a13-98ce-a9aac4bfb297 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:26.581906 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788171ba-04a9-4a13-98ce-a9aac4bfb297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:26.583591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-788171ba-04a9-4a13-98ce-a9aac4bfb297 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:26.591815 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:51:26.709107 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788171ba-04a9-4a13-98ce-a9aac4bfb297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:51:26.710571 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788171ba-04a9-4a13-98ce-a9aac4bfb297 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:51:26.728741 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 653/1322] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:51:26 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:51:29.866066 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b324866-0fc8-4426-8b6c-5c8b5bfc5f54 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:29.866626 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b324866-0fc8-4426-8b6c-5c8b5bfc5f54 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:29.866626 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b324866-0fc8-4426-8b6c-5c8b5bfc5f54 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:29.868132 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 670/1323] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:51:29.873701 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff9b0f-e8b9-4ba1-9b11-a405fe6b9fd9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:29.874152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff9b0f-e8b9-4ba1-9b11-a405fe6b9fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:29.874445 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff9b0f-e8b9-4ba1-9b11-a405fe6b9fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:29.947735 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d7ff9b0f-e8b9-4ba1-9b11-a405fe6b9fd9 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:51:29.948984 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d7ff9b0f-e8b9-4ba1-9b11-a405fe6b9fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:30.016137 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:51:30.061974 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 654/1324] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:51:30.099760 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b324866-0fc8-4426-8b6c-5c8b5bfc5f54 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:51:30.101403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-0b324866-0fc8-4426-8b6c-5c8b5bfc5f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:30.244265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b324866-0fc8-4426-8b6c-5c8b5bfc5f54 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:51:30.257834 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:51:30.357538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07545daa-7c93-40b2-b99a-09c178dd357d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:51:30.358166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07545daa-7c93-40b2-b99a-09c178dd357d service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:30.358447 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07545daa-7c93-40b2-b99a-09c178dd357d service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:30.358689 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07545daa-7c93-40b2-b99a-09c178dd357d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:30.360394 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-07545daa-7c93-40b2-b99a-09c178dd357d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:30.471485 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07545daa-7c93-40b2-b99a-09c178dd357d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:51:30.472974 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07545daa-7c93-40b2-b99a-09c178dd357d service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:51:30.492355 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 671/1325] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:51:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 04:51:32.604156 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d0922-3b10-4613-8110-92e4b248efc2 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:32.604734 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d0922-3b10-4613-8110-92e4b248efc2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:32.604734 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d0922-3b10-4613-8110-92e4b248efc2 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:32.605957 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 655/1326] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:51:32.611756 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b25e4-01b8-45be-bb92-7bc739c01fe6 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:32.612237 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b25e4-01b8-45be-bb92-7bc739c01fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:32.612433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b25e4-01b8-45be-bb92-7bc739c01fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:32.666741 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:51:32.687243 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-ef5b25e4-01b8-45be-bb92-7bc739c01fe6 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:51:32.689890 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-ef5b25e4-01b8-45be-bb92-7bc739c01fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:32.789990 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 672/1327] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:51:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:51:32.826809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d0922-3b10-4613-8110-92e4b248efc2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:51:32.828393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-da8d0922-3b10-4613-8110-92e4b248efc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:32.941091 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d0922-3b10-4613-8110-92e4b248efc2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:51:33.030147 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91615ab3-d22f-4665-9c09-4d340cb3341f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:51:33.030795 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91615ab3-d22f-4665-9c09-4d340cb3341f service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:33.031089 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91615ab3-d22f-4665-9c09-4d340cb3341f service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:33.031354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91615ab3-d22f-4665-9c09-4d340cb3341f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:33.033054 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-91615ab3-d22f-4665-9c09-4d340cb3341f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:33.052266 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:51:33.150013 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91615ab3-d22f-4665-9c09-4d340cb3341f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:51:33.151451 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91615ab3-d22f-4665-9c09-4d340cb3341f service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:51:33.179092 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 656/1328] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:51:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:51:35.232068 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b36fc2-52cf-4870-8309-d8f777e1a19d None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:35.232673 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b36fc2-52cf-4870-8309-d8f777e1a19d None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:35.232673 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b36fc2-52cf-4870-8309-d8f777e1a19d None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:35.234100 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 673/1329] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:51:35.240212 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58534935-dda9-4985-8ff3-8cfa7d8b694d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:35.240815 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58534935-dda9-4985-8ff3-8cfa7d8b694d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:35.240815 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58534935-dda9-4985-8ff3-8cfa7d8b694d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:35.313413 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-58534935-dda9-4985-8ff3-8cfa7d8b694d None None] MFA Rules not processed for user `7b1c7d019a3b49b79a0a9a3a4aea88f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:51:35.314631 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-58534935-dda9-4985-8ff3-8cfa7d8b694d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:35.387372 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:51:35.425189 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 657/1330] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:51:35 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:51:35.463337 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b36fc2-52cf-4870-8309-d8f777e1a19d None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:51:35.464937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-36b36fc2-52cf-4870-8309-d8f777e1a19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:35.573136 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b36fc2-52cf-4870-8309-d8f777e1a19d None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:51:35.595597 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:51:35.653429 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ae1f9-0cf7-4255-90ad-7deadfafbfeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:51:35.653943 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ae1f9-0cf7-4255-90ad-7deadfafbfeb service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:35.654211 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ae1f9-0cf7-4255-90ad-7deadfafbfeb service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:35.654480 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ae1f9-0cf7-4255-90ad-7deadfafbfeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:35.656332 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-3d9ae1f9-0cf7-4255-90ad-7deadfafbfeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:35.921050 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ae1f9-0cf7-4255-90ad-7deadfafbfeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:51:35.922539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ae1f9-0cf7-4255-90ad-7deadfafbfeb service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:51:35.946061 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 674/1331] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:51:35 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:51:38.259307 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08475b-bdbe-474e-90f6-30ef7b641267 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:38.260290 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08475b-bdbe-474e-90f6-30ef7b641267 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:38.260290 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08475b-bdbe-474e-90f6-30ef7b641267 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:38.262434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 658/1332] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:51:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:51:38.268756 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1066ac-0298-4485-bb81-bbf3e8de5393 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:38.269704 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1066ac-0298-4485-bb81-bbf3e8de5393 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:38.269704 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1066ac-0298-4485-bb81-bbf3e8de5393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:38.345961 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:51:38.349490 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-ea1066ac-0298-4485-bb81-bbf3e8de5393 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:51:38.351343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-ea1066ac-0298-4485-bb81-bbf3e8de5393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:38.450823 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 675/1333] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:51:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:51:38.486691 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb08475b-bdbe-474e-90f6-30ef7b641267 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:51:38.488299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bb08475b-bdbe-474e-90f6-30ef7b641267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:38.606674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb08475b-bdbe-474e-90f6-30ef7b641267 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:51:38.688439 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c74d4-b242-4a4f-bd16-4a25fc730e99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:51:38.689075 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c74d4-b242-4a4f-bd16-4a25fc730e99 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:38.689331 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c74d4-b242-4a4f-bd16-4a25fc730e99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:38.689645 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c74d4-b242-4a4f-bd16-4a25fc730e99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:38.691299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cb6c74d4-b242-4a4f-bd16-4a25fc730e99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:38.722329 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:51:38.816899 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c74d4-b242-4a4f-bd16-4a25fc730e99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:51:38.818370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c74d4-b242-4a4f-bd16-4a25fc730e99 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:51:38.837836 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 659/1334] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:51:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:51:40.924036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda6136-72bd-4579-9802-c2c33754e148 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:40.924646 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda6136-72bd-4579-9802-c2c33754e148 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:40.924646 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda6136-72bd-4579-9802-c2c33754e148 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:40.925976 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 676/1335] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:51:40.931589 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61ec0e-3b26-4c2d-a7c7-54f462e44bc8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:40.932119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61ec0e-3b26-4c2d-a7c7-54f462e44bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:40.932211 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61ec0e-3b26-4c2d-a7c7-54f462e44bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:41.011299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7a61ec0e-3b26-4c2d-a7c7-54f462e44bc8 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:51:41.012657 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7a61ec0e-3b26-4c2d-a7c7-54f462e44bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:41.124306 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:51:41.149532 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 660/1336] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:51:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:51:41.190184 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda6136-72bd-4579-9802-c2c33754e148 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:51:41.191782 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-5cda6136-72bd-4579-9802-c2c33754e148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:41.301319 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda6136-72bd-4579-9802-c2c33754e148 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:51:41.331061 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:51:41.393149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8f0db-6551-48fa-995c-b1a6a8a644ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:51:41.394149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8f0db-6551-48fa-995c-b1a6a8a644ce service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:41.394604 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8f0db-6551-48fa-995c-b1a6a8a644ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:41.395136 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8f0db-6551-48fa-995c-b1a6a8a644ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:41.398239 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-9be8f0db-6551-48fa-995c-b1a6a8a644ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:41.554917 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be8f0db-6551-48fa-995c-b1a6a8a644ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:51:41.556676 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be8f0db-6551-48fa-995c-b1a6a8a644ce service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:51:41.578055 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 677/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:51:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:51:43.982999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8bfbf-a77e-4f60-ac62-4daed49818f1 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:43.983797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8bfbf-a77e-4f60-ac62-4daed49818f1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:43.983797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8bfbf-a77e-4f60-ac62-4daed49818f1 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:43.985164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 661/1338] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:51:43.991830 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88725c7-327a-43e2-bd84-81c8f08aaf6b None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:43.992458 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88725c7-327a-43e2-bd84-81c8f08aaf6b None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:43.992571 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88725c7-327a-43e2-bd84-81c8f08aaf6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:44.082457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-e88725c7-327a-43e2-bd84-81c8f08aaf6b None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:51:44.083773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-e88725c7-327a-43e2-bd84-81c8f08aaf6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:44.094658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:51:44.189078 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 678/1339] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:51:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:51:44.225883 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8bfbf-a77e-4f60-ac62-4daed49818f1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:51:44.228248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a5a8bfbf-a77e-4f60-ac62-4daed49818f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:44.352341 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8bfbf-a77e-4f60-ac62-4daed49818f1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:51:44.433109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1182477-c6eb-49f9-a1a8-6b04c8ceb156 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:51:44.433743 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1182477-c6eb-49f9-a1a8-6b04c8ceb156 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:44.434029 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1182477-c6eb-49f9-a1a8-6b04c8ceb156 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:44.434298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1182477-c6eb-49f9-a1a8-6b04c8ceb156 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:44.435958 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f1182477-c6eb-49f9-a1a8-6b04c8ceb156 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:44.475165 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:51:44.555881 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1182477-c6eb-49f9-a1a8-6b04c8ceb156 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:51:44.557244 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1182477-c6eb-49f9-a1a8-6b04c8ceb156 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:51:44.577760 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 662/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:51:44 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:51:48.715176 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814c863-cc61-4302-a158-cf8c638d7304 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:48.715742 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814c863-cc61-4302-a158-cf8c638d7304 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:48.715742 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814c863-cc61-4302-a158-cf8c638d7304 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:48.717152 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 679/1341] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:51:48.722886 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb565417-457b-4dea-a5b0-3ada3a9033cf None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:48.723493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb565417-457b-4dea-a5b0-3ada3a9033cf None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:48.723493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb565417-457b-4dea-a5b0-3ada3a9033cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:48.808947 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-cb565417-457b-4dea-a5b0-3ada3a9033cf None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:51:48.810308 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cb565417-457b-4dea-a5b0-3ada3a9033cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:48.913959 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:51:48.927265 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 663/1342] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:51:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:51:48.966528 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814c863-cc61-4302-a158-cf8c638d7304 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:51:48.968189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-f814c863-cc61-4302-a158-cf8c638d7304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:49.080772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814c863-cc61-4302-a158-cf8c638d7304 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:51:49.131019 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:51:49.167697 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cedc7a3-c87d-4628-b8f8-f378da9ea1cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:51:49.168323 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedc7a3-c87d-4628-b8f8-f378da9ea1cf service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:49.168610 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedc7a3-c87d-4628-b8f8-f378da9ea1cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:49.168882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedc7a3-c87d-4628-b8f8-f378da9ea1cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:49.170522 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-8cedc7a3-c87d-4628-b8f8-f378da9ea1cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:49.282082 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cedc7a3-c87d-4628-b8f8-f378da9ea1cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:51:49.283530 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cedc7a3-c87d-4628-b8f8-f378da9ea1cf service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:51:49.302548 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 680/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:51:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:51:51.939237 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe5bce8-3c9c-4814-86e7-96557e0954cc None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:51.939889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe5bce8-3c9c-4814-86e7-96557e0954cc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:51.939889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe5bce8-3c9c-4814-86e7-96557e0954cc None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:51.941202 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 664/1344] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:51:51.946829 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbe8d5-607e-460a-a5e4-264951bf87f5 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:51.947291 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbe8d5-607e-460a-a5e4-264951bf87f5 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:51.947450 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbe8d5-607e-460a-a5e4-264951bf87f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:52.018616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-48dbe8d5-607e-460a-a5e4-264951bf87f5 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:51:52.019943 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-48dbe8d5-607e-460a-a5e4-264951bf87f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:52.042126 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:51:52.133662 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 681/1345] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:51:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:51:52.172511 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe5bce8-3c9c-4814-86e7-96557e0954cc None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:51:52.174060 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2fe5bce8-3c9c-4814-86e7-96557e0954cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:52.286918 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe5bce8-3c9c-4814-86e7-96557e0954cc None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:51:52.367435 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ab6bf-1bf3-474c-b003-3236baa4d1bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:51:52.368109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ab6bf-1bf3-474c-b003-3236baa4d1bf service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:52.368393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ab6bf-1bf3-474c-b003-3236baa4d1bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:52.368672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ab6bf-1bf3-474c-b003-3236baa4d1bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:52.370485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8d7ab6bf-1bf3-474c-b003-3236baa4d1bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:52.421122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:51:52.493715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7ab6bf-1bf3-474c-b003-3236baa4d1bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:51:52.495164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7ab6bf-1bf3-474c-b003-3236baa4d1bf service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:51:52.516209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 665/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:51:52 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:51:56.666566 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4ed66-49ee-48ad-ba1a-933362e12493 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:56.667253 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4ed66-49ee-48ad-ba1a-933362e12493 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:56.667253 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4ed66-49ee-48ad-ba1a-933362e12493 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:56.668759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 682/1347] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:51:56.674145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d78309-12e7-46dc-995e-9f71e41fd670 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:56.674629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d78309-12e7-46dc-995e-9f71e41fd670 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:56.674723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d78309-12e7-46dc-995e-9f71e41fd670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:56.750733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-10d78309-12e7-46dc-995e-9f71e41fd670 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:51:56.752131 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-10d78309-12e7-46dc-995e-9f71e41fd670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:56.853678 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:51:56.859906 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 666/1348] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:51:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:51:56.899019 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4ed66-49ee-48ad-ba1a-933362e12493 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:51:56.902887 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a5c4ed66-49ee-48ad-ba1a-933362e12493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:57.025827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4ed66-49ee-48ad-ba1a-933362e12493 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:51:57.084209 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:51:57.120551 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a813379-085b-4147-ad09-da382cf7f323 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:51:57.121307 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a813379-085b-4147-ad09-da382cf7f323 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:51:57.122294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a813379-085b-4147-ad09-da382cf7f323 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:51:57.122294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a813379-085b-4147-ad09-da382cf7f323 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:51:57.124281 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-7a813379-085b-4147-ad09-da382cf7f323 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:51:57.242507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a813379-085b-4147-ad09-da382cf7f323 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:51:57.244016 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a813379-085b-4147-ad09-da382cf7f323 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:51:57.265840 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 683/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:51:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 04:52:02.155224 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83c680-0185-4653-8910-9e6f27a16931 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:02.155828 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83c680-0185-4653-8910-9e6f27a16931 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:02.155828 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83c680-0185-4653-8910-9e6f27a16931 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:02.158432 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 667/1350] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jul 26 04:52:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:52:02.179992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc9eb0-a5ae-470a-9457-8ffa8a276e52 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:02.179992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc9eb0-a5ae-470a-9457-8ffa8a276e52 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:02.179992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc9eb0-a5ae-470a-9457-8ffa8a276e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:02.293403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-e7cc9eb0-a5ae-470a-9457-8ffa8a276e52 None None] MFA Rules not processed for user `5049dba7ffde47b8af3e6fd55b5703e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:52:02.294620 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-e7cc9eb0-a5ae-470a-9457-8ffa8a276e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:02.316025 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:02.421371 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 684/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jul 26 04:52:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:52:06.469489 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc805afe-3e46-44f0-98b6-d73fd9235c6d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:06.470349 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc805afe-3e46-44f0-98b6-d73fd9235c6d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:06.470796 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc805afe-3e46-44f0-98b6-d73fd9235c6d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:06.472472 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 668/1352] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jul 26 04:52:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:52:06.478050 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617fcc7-29d8-4e31-b9e0-4d002d3ea835 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:06.478558 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617fcc7-29d8-4e31-b9e0-4d002d3ea835 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:06.478558 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617fcc7-29d8-4e31-b9e0-4d002d3ea835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:06.562842 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-0617fcc7-29d8-4e31-b9e0-4d002d3ea835 None None] MFA Rules not processed for user `5049dba7ffde47b8af3e6fd55b5703e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:52:06.564137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-0617fcc7-29d8-4e31-b9e0-4d002d3ea835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:06.640402 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:06.685297 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 685/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jul 26 04:52:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:52:08.260612 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10a0ee-6182-4483-9e34-32dae6b3bf83 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:08.261192 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10a0ee-6182-4483-9e34-32dae6b3bf83 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:08.261372 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10a0ee-6182-4483-9e34-32dae6b3bf83 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:08.263086 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 669/1354] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jul 26 04:52:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:52:08.268577 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a2868-fd92-4671-9638-25060b240dbe None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:08.269034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a2868-fd92-4671-9638-25060b240dbe None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:08.269122 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a2868-fd92-4671-9638-25060b240dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:08.348584 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-1e7a2868-fd92-4671-9638-25060b240dbe None None] MFA Rules not processed for user `5049dba7ffde47b8af3e6fd55b5703e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:52:08.349808 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-1e7a2868-fd92-4671-9638-25060b240dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:08.467577 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 686/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jul 26 04:52:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:52:11.844259 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc6cca-0f9f-4ded-885a-e183a90af663 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:11.845216 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc6cca-0f9f-4ded-885a-e183a90af663 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:11.845645 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc6cca-0f9f-4ded-885a-e183a90af663 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:11.847672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 670/1356] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:52:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:52:11.855739 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ead97c-4ce3-4dd9-84f0-9d7becf77df0 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:11.857499 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ead97c-4ce3-4dd9-84f0-9d7becf77df0 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:11.858032 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ead97c-4ce3-4dd9-84f0-9d7becf77df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:11.911278 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:11.961302 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-82ead97c-4ce3-4dd9-84f0-9d7becf77df0 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:52:11.962564 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-82ead97c-4ce3-4dd9-84f0-9d7becf77df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:12.075692 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 687/1357] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:52:12.135367 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bb40c-f5a5-4efe-84f5-c00b22b4c689 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:12.135629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bb40c-f5a5-4efe-84f5-c00b22b4c689 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:12.135926 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bb40c-f5a5-4efe-84f5-c00b22b4c689 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:12.137300 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 671/1358] 10.222.0.17 () {54 vars in 962 bytes} [Tue Jul 26 04:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:52:12.147976 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17a6aa-2288-4c49-965c-521a38e77f8c None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:12.148430 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17a6aa-2288-4c49-965c-521a38e77f8c None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:12.148810 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17a6aa-2288-4c49-965c-521a38e77f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:12.245119 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-7a17a6aa-2288-4c49-965c-521a38e77f8c None None] MFA Rules not processed for user `2faeed9002ee4154bc980c146cd06bcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:52:12.246521 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-7a17a6aa-2288-4c49-965c-521a38e77f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:12.309347 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:12.396532 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 688/1359] 10.222.0.17 () {58 vars in 1059 bytes} [Tue Jul 26 04:52:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:52:12.405513 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8bb40c-f5a5-4efe-84f5-c00b22b4c689 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:52:12.407231 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6a8bb40c-f5a5-4efe-84f5-c00b22b4c689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:12.565686 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8bb40c-f5a5-4efe-84f5-c00b22b4c689 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:52:12.647034 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:12.690055 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e18974-4197-4886-bcab-4f2e69f84149 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:52:12.690694 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e18974-4197-4886-bcab-4f2e69f84149 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:12.691391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e18974-4197-4886-bcab-4f2e69f84149 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:12.691664 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e18974-4197-4886-bcab-4f2e69f84149 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:12.693330 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-46e18974-4197-4886-bcab-4f2e69f84149 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:12.791337 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e18974-4197-4886-bcab-4f2e69f84149 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:52:12.793671 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e18974-4197-4886-bcab-4f2e69f84149 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:52:12.812752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 672/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:52:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 04:52:15.049258 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d014641-d395-4bf4-b0a5-28283ecf3cac None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:15.049806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d014641-d395-4bf4-b0a5-28283ecf3cac None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:15.049806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d014641-d395-4bf4-b0a5-28283ecf3cac None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:15.051603 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 689/1361] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:52:15.057475 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c5185-ce61-42e9-b9d7-0c171d76a7c0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:15.058334 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c5185-ce61-42e9-b9d7-0c171d76a7c0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:15.058784 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c5185-ce61-42e9-b9d7-0c171d76a7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:15.135886 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-511c5185-ce61-42e9-b9d7-0c171d76a7c0 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:52:15.137324 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-511c5185-ce61-42e9-b9d7-0c171d76a7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:15.179670 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:15.258442 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 673/1362] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:52:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:52:15.306868 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d014641-d395-4bf4-b0a5-28283ecf3cac None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:52:15.308571 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-2d014641-d395-4bf4-b0a5-28283ecf3cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:15.506441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d014641-d395-4bf4-b0a5-28283ecf3cac None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:52:15.520432 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:15.635895 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d0366-43f5-445c-923c-dc6eaf5bf6bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:52:15.636528 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d0366-43f5-445c-923c-dc6eaf5bf6bb service nova] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:15.636805 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d0366-43f5-445c-923c-dc6eaf5bf6bb service nova] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:15.637079 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d0366-43f5-445c-923c-dc6eaf5bf6bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:15.638881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-389d0366-43f5-445c-923c-dc6eaf5bf6bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:15.751897 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d0366-43f5-445c-923c-dc6eaf5bf6bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:52:15.753541 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d0366-43f5-445c-923c-dc6eaf5bf6bb service nova] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:52:15.772578 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 690/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:52:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:52:18.108630 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e9d2b-76b1-4947-a67a-2a3f9229c65b None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:18.109239 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e9d2b-76b1-4947-a67a-2a3f9229c65b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:18.109239 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e9d2b-76b1-4947-a67a-2a3f9229c65b None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:18.110535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 674/1364] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:52:18.116437 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a562b-2a00-4e41-93ac-75e62fa48feb None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:18.116935 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a562b-2a00-4e41-93ac-75e62fa48feb None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:18.117061 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a562b-2a00-4e41-93ac-75e62fa48feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:18.138915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:18.206044 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-e28a562b-2a00-4e41-93ac-75e62fa48feb None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:52:18.207963 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-e28a562b-2a00-4e41-93ac-75e62fa48feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:18.343988 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 691/1365] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:52:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:52:18.382010 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8117c3c-310a-416a-805b-fdd824006d18 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:18.382350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8117c3c-310a-416a-805b-fdd824006d18 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:18.382580 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8117c3c-310a-416a-805b-fdd824006d18 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:18.385264 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 675/1366] 10.222.0.17 () {54 vars in 962 bytes} [Tue Jul 26 04:52:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:52:18.394473 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a493a2-e26d-4dad-bc1f-ff4cd6d3f483 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:18.394732 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a493a2-e26d-4dad-bc1f-ff4cd6d3f483 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:18.395020 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a493a2-e26d-4dad-bc1f-ff4cd6d3f483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:18.488329 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-62a493a2-e26d-4dad-bc1f-ff4cd6d3f483 None None] MFA Rules not processed for user `2faeed9002ee4154bc980c146cd06bcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:52:18.490039 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-62a493a2-e26d-4dad-bc1f-ff4cd6d3f483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:18.532479 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:18.624612 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 692/1367] 10.222.0.17 () {58 vars in 1059 bytes} [Tue Jul 26 04:52:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:52:18.632511 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8117c3c-310a-416a-805b-fdd824006d18 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:52:18.634435 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c8117c3c-310a-416a-805b-fdd824006d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:18.813484 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8117c3c-310a-416a-805b-fdd824006d18 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:52:18.836398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:18.942852 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc8b7d-350d-4686-8231-98777f0ff1cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:52:18.943402 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc8b7d-350d-4686-8231-98777f0ff1cb service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:18.943846 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc8b7d-350d-4686-8231-98777f0ff1cb service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:18.944118 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc8b7d-350d-4686-8231-98777f0ff1cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:18.945793 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-14cc8b7d-350d-4686-8231-98777f0ff1cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:19.040559 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc8b7d-350d-4686-8231-98777f0ff1cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:52:19.042662 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc8b7d-350d-4686-8231-98777f0ff1cb service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:52:19.074180 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 676/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:52:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:52:20.062391 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453ae32-5e72-4bb6-b4e4-09519758668f None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:20.063918 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453ae32-5e72-4bb6-b4e4-09519758668f None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:20.064320 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453ae32-5e72-4bb6-b4e4-09519758668f None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:20.066241 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 693/1369] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:52:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:52:20.073032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d6c69-b94d-4757-ab5e-279d0fe503a8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:20.074421 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d6c69-b94d-4757-ab5e-279d0fe503a8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:20.075187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d6c69-b94d-4757-ab5e-279d0fe503a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:20.155776 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:20.208948 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d88d6c69-b94d-4757-ab5e-279d0fe503a8 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:52:20.210372 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d88d6c69-b94d-4757-ab5e-279d0fe503a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:20.348986 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 677/1370] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:52:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:52:20.721663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b4b51-0b14-446a-968b-5eb114f06cf1 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:20.721663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b4b51-0b14-446a-968b-5eb114f06cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:20.721663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b4b51-0b14-446a-968b-5eb114f06cf1 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:20.721663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 694/1371] 10.222.0.17 () {54 vars in 964 bytes} [Tue Jul 26 04:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:52:20.726343 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199cf3b-ce50-41f6-bc3a-0ed82fe563a1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:20.726343 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199cf3b-ce50-41f6-bc3a-0ed82fe563a1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:20.726343 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199cf3b-ce50-41f6-bc3a-0ed82fe563a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:20.817438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2199cf3b-ce50-41f6-bc3a-0ed82fe563a1 None None] MFA Rules not processed for user `79ef3d83ac454fac9f890374cd22507a`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:52:20.818687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2199cf3b-ce50-41f6-bc3a-0ed82fe563a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:20.874723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:20.917235 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 678/1372] 10.222.0.17 () {58 vars in 1061 bytes} [Tue Jul 26 04:52:20 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 04:52:20.919368 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b4b51-0b14-446a-968b-5eb114f06cf1 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:52:20.922848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-045b4b51-0b14-446a-968b-5eb114f06cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:21.132271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b4b51-0b14-446a-968b-5eb114f06cf1 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:52:21.180918 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:21.222202 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2993c5e-0462-45d8-b17b-b1045b95ed53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:52:21.222977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2993c5e-0462-45d8-b17b-b1045b95ed53 service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:21.223233 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2993c5e-0462-45d8-b17b-b1045b95ed53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:21.223765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2993c5e-0462-45d8-b17b-b1045b95ed53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:21.225315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a2993c5e-0462-45d8-b17b-b1045b95ed53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:21.354568 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2993c5e-0462-45d8-b17b-b1045b95ed53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:52:21.354568 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2993c5e-0462-45d8-b17b-b1045b95ed53 service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:52:21.393574 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 695/1373] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:52:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:52:21.862134 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ddca8-7efc-4d95-8773-68aa0a44173e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:21.862955 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ddca8-7efc-4d95-8773-68aa0a44173e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:21.862955 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ddca8-7efc-4d95-8773-68aa0a44173e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:21.864661 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 679/1374] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:52:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:52:21.870927 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1d367-06ef-4fa2-bb90-444c975b0281 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:21.871413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1d367-06ef-4fa2-bb90-444c975b0281 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:21.871955 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1d367-06ef-4fa2-bb90-444c975b0281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:21.979792 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-b7f1d367-06ef-4fa2-bb90-444c975b0281 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:52:21.981208 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-b7f1d367-06ef-4fa2-bb90-444c975b0281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:22.009443 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:22.116146 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 696/1375] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:52:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:52:22.191027 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ddca8-7efc-4d95-8773-68aa0a44173e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:52:22.193565 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4f5ddca8-7efc-4d95-8773-68aa0a44173e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:22.409587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ddca8-7efc-4d95-8773-68aa0a44173e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:52:22.478860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:22.554366 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a3510-b670-424a-899c-07b92ef841a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:52:22.555168 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a3510-b670-424a-899c-07b92ef841a1 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:22.555585 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a3510-b670-424a-899c-07b92ef841a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:22.555969 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a3510-b670-424a-899c-07b92ef841a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:22.557968 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-094a3510-b670-424a-899c-07b92ef841a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:22.666265 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a3510-b670-424a-899c-07b92ef841a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:52:22.668003 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a3510-b670-424a-899c-07b92ef841a1 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:52:22.692033 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 680/1376] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:52:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 04:52:24.882519 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04c6ce-cf72-4c31-b5bd-0d19ae20a778 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:24.883133 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04c6ce-cf72-4c31-b5bd-0d19ae20a778 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:24.883661 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04c6ce-cf72-4c31-b5bd-0d19ae20a778 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:24.886352 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 697/1377] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:52:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:52:24.892561 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af7fda-dc4a-4d26-b3dc-26b74b46ae9b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:24.893242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af7fda-dc4a-4d26-b3dc-26b74b46ae9b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:24.893242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af7fda-dc4a-4d26-b3dc-26b74b46ae9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:24.975149 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-98af7fda-dc4a-4d26-b3dc-26b74b46ae9b None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:52:24.977173 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-98af7fda-dc4a-4d26-b3dc-26b74b46ae9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:24.989250 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:25.110522 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 681/1378] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:52:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 04:52:25.158145 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04c6ce-cf72-4c31-b5bd-0d19ae20a778 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:52:25.159852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-ca04c6ce-cf72-4c31-b5bd-0d19ae20a778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:25.330423 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04c6ce-cf72-4c31-b5bd-0d19ae20a778 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:52:25.334379 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:25.515490 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65adcc8-9117-4ea4-83f3-54f48a6191eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:52:25.518224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65adcc8-9117-4ea4-83f3-54f48a6191eb service nova] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:25.518734 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65adcc8-9117-4ea4-83f3-54f48a6191eb service nova] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:25.519248 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65adcc8-9117-4ea4-83f3-54f48a6191eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:25.522488 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-d65adcc8-9117-4ea4-83f3-54f48a6191eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:25.654034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65adcc8-9117-4ea4-83f3-54f48a6191eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:52:25.656206 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65adcc8-9117-4ea4-83f3-54f48a6191eb service nova] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:52:25.679909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 698/1379] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:52:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:52:28.041129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051981ba-ff89-4f16-882c-cb60fcf8ebbb None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:28.041939 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051981ba-ff89-4f16-882c-cb60fcf8ebbb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:28.041939 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051981ba-ff89-4f16-882c-cb60fcf8ebbb None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:28.047803 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 682/1380] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:52:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:52:28.052051 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2584e10-e426-4754-8ab0-9aba180c97b0 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:28.053468 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2584e10-e426-4754-8ab0-9aba180c97b0 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:28.053811 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2584e10-e426-4754-8ab0-9aba180c97b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:28.067854 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:28.147431 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-c2584e10-e426-4754-8ab0-9aba180c97b0 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:52:28.148793 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c2584e10-e426-4754-8ab0-9aba180c97b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:28.283788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 699/1381] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:52:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:52:28.337901 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051981ba-ff89-4f16-882c-cb60fcf8ebbb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:52:28.339683 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-051981ba-ff89-4f16-882c-cb60fcf8ebbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:28.598428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:28.651368 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051981ba-ff89-4f16-882c-cb60fcf8ebbb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:52:28.973908 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7f919-f1ae-4147-9ef4-913e474a1a9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:52:28.977308 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7f919-f1ae-4147-9ef4-913e474a1a9d service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:28.977814 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7f919-f1ae-4147-9ef4-913e474a1a9d service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:28.978328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7f919-f1ae-4147-9ef4-913e474a1a9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:28.982606 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5ab7f919-f1ae-4147-9ef4-913e474a1a9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:29.210557 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7f919-f1ae-4147-9ef4-913e474a1a9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:52:29.214737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7f919-f1ae-4147-9ef4-913e474a1a9d service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:52:29.240205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:29.259602 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 683/1382] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:52:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 922 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:52:32.072910 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdaaa9e-4898-474f-90c0-61b1129a3f07 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:32.074110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdaaa9e-4898-474f-90c0-61b1129a3f07 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:32.074110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdaaa9e-4898-474f-90c0-61b1129a3f07 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:32.075347 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 700/1383] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:52:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:52:32.083078 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10736f64-c86e-4dce-b2fa-0f7e3c7ffbf2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:32.084085 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10736f64-c86e-4dce-b2fa-0f7e3c7ffbf2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:32.084381 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10736f64-c86e-4dce-b2fa-0f7e3c7ffbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:32.189776 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-10736f64-c86e-4dce-b2fa-0f7e3c7ffbf2 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:52:32.191144 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-10736f64-c86e-4dce-b2fa-0f7e3c7ffbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:32.352816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 684/1384] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:52:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:52:32.434505 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdaaa9e-4898-474f-90c0-61b1129a3f07 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:52:32.436214 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-5fdaaa9e-4898-474f-90c0-61b1129a3f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:32.580836 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:32.690930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdaaa9e-4898-474f-90c0-61b1129a3f07 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:52:32.895918 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d861933-532e-4b2e-b9cc-8cbbdd32a42e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:52:32.899240 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d861933-532e-4b2e-b9cc-8cbbdd32a42e service nova] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:32.899969 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d861933-532e-4b2e-b9cc-8cbbdd32a42e service nova] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:32.900802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d861933-532e-4b2e-b9cc-8cbbdd32a42e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:32.905949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-7d861933-532e-4b2e-b9cc-8cbbdd32a42e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:33.010101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:33.073468 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d861933-532e-4b2e-b9cc-8cbbdd32a42e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:52:33.076388 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d861933-532e-4b2e-b9cc-8cbbdd32a42e service nova] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:52:33.106428 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 701/1385] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:52:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 672 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:52:35.487488 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056ed82-c96b-4747-96b6-922aaa1ee78f None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:35.488253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056ed82-c96b-4747-96b6-922aaa1ee78f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:35.488253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056ed82-c96b-4747-96b6-922aaa1ee78f None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:35.489538 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 685/1386] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:52:35.495768 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7bb7a5-fb06-42b6-9b9b-1a1b2be21175 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:35.496582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7bb7a5-fb06-42b6-9b9b-1a1b2be21175 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:35.496582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7bb7a5-fb06-42b6-9b9b-1a1b2be21175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:35.591819 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-3b7bb7a5-fb06-42b6-9b9b-1a1b2be21175 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:52:35.593317 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-3b7bb7a5-fb06-42b6-9b9b-1a1b2be21175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:35.732769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 702/1387] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:52:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:52:35.786906 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056ed82-c96b-4747-96b6-922aaa1ee78f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:52:35.788860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6056ed82-c96b-4747-96b6-922aaa1ee78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:35.890135 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:35.987280 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056ed82-c96b-4747-96b6-922aaa1ee78f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:52:36.148099 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50cfdc-d1e8-4aac-8264-b04fdf1a0f68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:52:36.149307 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50cfdc-d1e8-4aac-8264-b04fdf1a0f68 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:36.149723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50cfdc-d1e8-4aac-8264-b04fdf1a0f68 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:36.150173 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50cfdc-d1e8-4aac-8264-b04fdf1a0f68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:36.152297 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2e50cfdc-d1e8-4aac-8264-b04fdf1a0f68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:36.207864 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:36.282666 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e50cfdc-d1e8-4aac-8264-b04fdf1a0f68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:52:36.285836 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e50cfdc-d1e8-4aac-8264-b04fdf1a0f68 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:52:36.317736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 686/1388] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:52:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:52:37.559342 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296fa5d2-fecb-4590-b2f1-95c85ca611fd None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:37.560321 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296fa5d2-fecb-4590-b2f1-95c85ca611fd None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:37.560321 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296fa5d2-fecb-4590-b2f1-95c85ca611fd None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:37.561728 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 703/1389] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:52:37.568601 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d20a04-246f-40b2-b6e0-2319af000e17 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:37.569432 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d20a04-246f-40b2-b6e0-2319af000e17 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:37.569599 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d20a04-246f-40b2-b6e0-2319af000e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:37.696452 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a2d20a04-246f-40b2-b6e0-2319af000e17 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:52:37.697852 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a2d20a04-246f-40b2-b6e0-2319af000e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:37.855545 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 687/1390] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:52:37.920242 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48896480-aecf-4913-b1fe-4785debd50b7 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:37.920734 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48896480-aecf-4913-b1fe-4785debd50b7 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:37.921220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48896480-aecf-4913-b1fe-4785debd50b7 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:37.923764 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 704/1391] 10.222.0.17 () {54 vars in 966 bytes} [Tue Jul 26 04:52:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:52:37.932367 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920475b7-0f74-4b06-95cd-0648be91f92f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:37.932694 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920475b7-0f74-4b06-95cd-0648be91f92f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:37.932930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920475b7-0f74-4b06-95cd-0648be91f92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:37.952176 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:38.033646 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-920475b7-0f74-4b06-95cd-0648be91f92f None None] MFA Rules not processed for user `6946a788d2ef49ff873ca8e81990527e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:52:38.034953 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-920475b7-0f74-4b06-95cd-0648be91f92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:38.132834 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 688/1392] 10.222.0.17 () {58 vars in 1063 bytes} [Tue Jul 26 04:52:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:52:38.141651 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48896480-aecf-4913-b1fe-4785debd50b7 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:52:38.143471 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-48896480-aecf-4913-b1fe-4785debd50b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:38.181120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:38.271622 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48896480-aecf-4913-b1fe-4785debd50b7 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:52:38.384933 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011a3f6-d712-48a8-88c6-94ebfc6b257c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6946a788d2ef49ff873ca8e81990527e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:52:38.386260 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011a3f6-d712-48a8-88c6-94ebfc6b257c service glance] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:38.386888 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011a3f6-d712-48a8-88c6-94ebfc6b257c service glance] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:38.387496 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011a3f6-d712-48a8-88c6-94ebfc6b257c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:38.391414 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a011a3f6-d712-48a8-88c6-94ebfc6b257c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:38.496116 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:38.535999 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011a3f6-d712-48a8-88c6-94ebfc6b257c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:52:38.537434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011a3f6-d712-48a8-88c6-94ebfc6b257c service glance] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:52:38.561159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 705/1393] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:52:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:52:38.915766 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5d22c-050c-4644-8f0c-1b8215ab643d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:38.916619 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5d22c-050c-4644-8f0c-1b8215ab643d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:38.916619 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5d22c-050c-4644-8f0c-1b8215ab643d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:38.920136 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 689/1394] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:52:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:52:38.927109 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776d4af-aee6-41ad-a691-6a3dfd12b013 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:38.927522 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776d4af-aee6-41ad-a691-6a3dfd12b013 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:38.927726 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776d4af-aee6-41ad-a691-6a3dfd12b013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:39.045958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-c776d4af-aee6-41ad-a691-6a3dfd12b013 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:52:39.048161 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c776d4af-aee6-41ad-a691-6a3dfd12b013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:39.239223 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 706/1395] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:52:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 04:52:39.292815 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5d22c-050c-4644-8f0c-1b8215ab643d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:52:39.294802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-89e5d22c-050c-4644-8f0c-1b8215ab643d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:39.449901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5037c-074a-4291-89b0-ee4f14e56db0 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:39.450342 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5037c-074a-4291-89b0-ee4f14e56db0 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:39.450805 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5037c-074a-4291-89b0-ee4f14e56db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:39.454589 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:39.514870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5d22c-050c-4644-8f0c-1b8215ab643d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:52:39.520933 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:39.552983 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-98e5037c-074a-4291-89b0-ee4f14e56db0 None None] MFA Rules not processed for user `3a8f3eb811ae40ad9cb64261f95a992e`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:52:39.554746 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-98e5037c-074a-4291-89b0-ee4f14e56db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:39.668178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53891b-9dde-4272-91d1-883cddb1535c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:52:39.669183 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53891b-9dde-4272-91d1-883cddb1535c service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:39.669641 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53891b-9dde-4272-91d1-883cddb1535c service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:39.670087 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53891b-9dde-4272-91d1-883cddb1535c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:39.673227 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5d53891b-9dde-4272-91d1-883cddb1535c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:39.680189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 707/1396] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jul 26 04:52:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 04:52:39.744799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe369f6-706a-4793-bf5a-2dbd45c54471 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:39.745176 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe369f6-706a-4793-bf5a-2dbd45c54471 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:39.745560 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe369f6-706a-4793-bf5a-2dbd45c54471 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:39.747081 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 708/1397] 10.222.0.17 () {54 vars in 943 bytes} [Tue Jul 26 04:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:52:39.755483 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0171a-91c9-448c-93aa-913db329ba99 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:39.756116 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0171a-91c9-448c-93aa-913db329ba99 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:39.756683 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0171a-91c9-448c-93aa-913db329ba99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:39.776311 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:39.805397 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d53891b-9dde-4272-91d1-883cddb1535c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:52:39.808409 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d53891b-9dde-4272-91d1-883cddb1535c service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:52:39.833708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 690/1398] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:52:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:52:39.873303 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-5fb0171a-91c9-448c-93aa-913db329ba99 None None] MFA Rules not processed for user `d5218319aaa5424882714bffcbeee730`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:52:39.875662 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-5fb0171a-91c9-448c-93aa-913db329ba99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:39.982250 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:40.047846 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 709/1399] 10.222.0.17 () {58 vars in 1040 bytes} [Tue Jul 26 04:52:39 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 04:52:40.061229 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53891b-9dde-4272-91d1-883cddb1535c service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:52:40.062857 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5d53891b-9dde-4272-91d1-883cddb1535c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:40.177043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53891b-9dde-4272-91d1-883cddb1535c service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:52:40.353190 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:40.356756 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08bcb73-2a76-4f7b-87e1-06df1c335316 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5218319aaa5424882714bffcbeee730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:52:40.357872 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bcb73-2a76-4f7b-87e1-06df1c335316 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:40.358557 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bcb73-2a76-4f7b-87e1-06df1c335316 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:40.358979 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bcb73-2a76-4f7b-87e1-06df1c335316 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:40.361593 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d08bcb73-2a76-4f7b-87e1-06df1c335316 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:40.487093 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08bcb73-2a76-4f7b-87e1-06df1c335316 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:52:40.489005 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08bcb73-2a76-4f7b-87e1-06df1c335316 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:52:40.499355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 691/1400] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jul 26 04:52:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 04:52:42.215922 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db6c05-dd1d-4c52-b4f1-f90040fb2ef4 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:42.216890 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db6c05-dd1d-4c52-b4f1-f90040fb2ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:42.216890 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db6c05-dd1d-4c52-b4f1-f90040fb2ef4 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:42.224109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3515f7-3eec-49a2-b4a5-74e0360f47df None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:42.224926 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3515f7-3eec-49a2-b4a5-74e0360f47df None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:42.224926 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3515f7-3eec-49a2-b4a5-74e0360f47df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:42.229554 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 710/1401] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:52:42 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:52:42.325555 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7d3515f7-3eec-49a2-b4a5-74e0360f47df None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:52:42.326910 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7d3515f7-3eec-49a2-b4a5-74e0360f47df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:42.454178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:42.466734 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 692/1402] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:52:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:52:42.515210 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db6c05-dd1d-4c52-b4f1-f90040fb2ef4 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:52:42.516823 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c8db6c05-dd1d-4c52-b4f1-f90040fb2ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:42.699633 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db6c05-dd1d-4c52-b4f1-f90040fb2ef4 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:52:42.749276 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:42.844500 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd64dfe7-99db-4b65-a734-f6294ad34f00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:52:42.845341 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64dfe7-99db-4b65-a734-f6294ad34f00 service nova] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:42.845628 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64dfe7-99db-4b65-a734-f6294ad34f00 service nova] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:42.845910 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64dfe7-99db-4b65-a734-f6294ad34f00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:42.847867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-fd64dfe7-99db-4b65-a734-f6294ad34f00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:42.967704 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd64dfe7-99db-4b65-a734-f6294ad34f00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:52:42.969394 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd64dfe7-99db-4b65-a734-f6294ad34f00 service nova] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:52:42.992717 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 711/1403] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:52:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:52:45.523903 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a23071-4e9a-4314-8ee5-46bc6418d9f7 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:45.524521 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a23071-4e9a-4314-8ee5-46bc6418d9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:45.524897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a23071-4e9a-4314-8ee5-46bc6418d9f7 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:45.527602 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 693/1404] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:52:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:52:45.535589 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb5a35-03cd-4ce8-8280-6ec8b0bfe649 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:45.536395 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb5a35-03cd-4ce8-8280-6ec8b0bfe649 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:45.536395 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb5a35-03cd-4ce8-8280-6ec8b0bfe649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:45.619443 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:45.636033 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-52fb5a35-03cd-4ce8-8280-6ec8b0bfe649 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:52:45.636838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-52fb5a35-03cd-4ce8-8280-6ec8b0bfe649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:45.819456 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 712/1405] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:52:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:52:45.882708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a23071-4e9a-4314-8ee5-46bc6418d9f7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:52:45.886122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c5a23071-4e9a-4314-8ee5-46bc6418d9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:46.226881 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a23071-4e9a-4314-8ee5-46bc6418d9f7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:52:46.361901 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:46.491452 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d2a33-2677-4f5d-9538-86fe760a8cf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:52:46.492535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d2a33-2677-4f5d-9538-86fe760a8cf6 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:46.493017 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d2a33-2677-4f5d-9538-86fe760a8cf6 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:46.493496 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d2a33-2677-4f5d-9538-86fe760a8cf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:46.496761 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-807d2a33-2677-4f5d-9538-86fe760a8cf6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:46.654826 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807d2a33-2677-4f5d-9538-86fe760a8cf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:52:46.657817 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807d2a33-2677-4f5d-9538-86fe760a8cf6 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:52:46.704177 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 694/1406] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:52:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 823 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 04:52:49.171509 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf92cd4-29ba-4874-92f7-424fd924985d None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:49.172124 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf92cd4-29ba-4874-92f7-424fd924985d None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:49.172124 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf92cd4-29ba-4874-92f7-424fd924985d None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:49.174175 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 713/1407] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:52:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:52:49.182279 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb07ce5-aa5d-4692-8ee9-04bf9d2c8ed6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:49.183191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb07ce5-aa5d-4692-8ee9-04bf9d2c8ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:49.183679 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb07ce5-aa5d-4692-8ee9-04bf9d2c8ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:49.279641 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3cb07ce5-aa5d-4692-8ee9-04bf9d2c8ed6 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:52:49.280931 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3cb07ce5-aa5d-4692-8ee9-04bf9d2c8ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:49.330450 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:49.443301 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 695/1408] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:52:49.494575 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf92cd4-29ba-4874-92f7-424fd924985d None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:52:49.496333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-8cf92cd4-29ba-4874-92f7-424fd924985d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:49.666129 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:49.703384 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf92cd4-29ba-4874-92f7-424fd924985d None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:52:49.886423 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891a7eb-3f11-47ba-bca2-cd9cf83e01d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:52:49.887180 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891a7eb-3f11-47ba-bca2-cd9cf83e01d2 service nova] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:52:49.887487 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891a7eb-3f11-47ba-bca2-cd9cf83e01d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:52:49.887805 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891a7eb-3f11-47ba-bca2-cd9cf83e01d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:52:49.889661 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-b891a7eb-3f11-47ba-bca2-cd9cf83e01d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:52:50.016889 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:52:50.021622 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b891a7eb-3f11-47ba-bca2-cd9cf83e01d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:52:50.030253 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b891a7eb-3f11-47ba-bca2-cd9cf83e01d2 service nova] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:52:50.050558 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 714/1409] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:52:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 04:52:59.440395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=30 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:00.045742 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=7 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:06.196136 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93593bf0-1dc7-44f5-9235-7837bdcaa14e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:06.196737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93593bf0-1dc7-44f5-9235-7837bdcaa14e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:06.197085 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93593bf0-1dc7-44f5-9235-7837bdcaa14e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:06.199852 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 696/1410] 10.222.0.17 () {54 vars in 918 bytes} [Tue Jul 26 04:53:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:53:06.209203 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81466d13-3b18-4f1a-807d-4039a42adf3b None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:06.209809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81466d13-3b18-4f1a-807d-4039a42adf3b None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:06.209809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81466d13-3b18-4f1a-807d-4039a42adf3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:06.306352 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-81466d13-3b18-4f1a-807d-4039a42adf3b None None] MFA Rules not processed for user `5049dba7ffde47b8af3e6fd55b5703e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:53:06.307907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-81466d13-3b18-4f1a-807d-4039a42adf3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:06.455051 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 715/1411] 10.222.0.17 () {58 vars in 1015 bytes} [Tue Jul 26 04:53:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:53:06.476526 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161312aa-a3d2-4791-99fa-d2315436644a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:06.476841 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161312aa-a3d2-4791-99fa-d2315436644a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:06.477108 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161312aa-a3d2-4791-99fa-d2315436644a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:06.478517 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 697/1412] 10.222.0.17 () {54 vars in 918 bytes} [Tue Jul 26 04:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:53:06.484734 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075d759-1436-455d-841d-4969e47da344 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:06.485021 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075d759-1436-455d-841d-4969e47da344 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:06.485286 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075d759-1436-455d-841d-4969e47da344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:06.600001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:06.605072 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-a075d759-1436-455d-841d-4969e47da344 None None] MFA Rules not processed for user `2faeed9002ee4154bc980c146cd06bcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:53:06.607128 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a075d759-1436-455d-841d-4969e47da344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:06.765410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 716/1413] 10.222.0.17 () {58 vars in 1015 bytes} [Tue Jul 26 04:53:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:53:09.800605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8115e3d-325a-47e3-9779-f2c372d3503a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:09.801492 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8115e3d-325a-47e3-9779-f2c372d3503a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:09.801492 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8115e3d-325a-47e3-9779-f2c372d3503a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:09.803097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 698/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 04:53:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:53:09.809696 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a1268-cb58-4644-bc64-078e3611f8de None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:09.810421 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a1268-cb58-4644-bc64-078e3611f8de None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:09.810421 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a1268-cb58-4644-bc64-078e3611f8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:09.889792 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-a53a1268-cb58-4644-bc64-078e3611f8de None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:53:09.890964 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a53a1268-cb58-4644-bc64-078e3611f8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:09.958200 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:10.012298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 717/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 04:53:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:53:10.062204 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8115e3d-325a-47e3-9779-f2c372d3503a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:53:10.065030 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e8115e3d-325a-47e3-9779-f2c372d3503a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:10.407674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8115e3d-325a-47e3-9779-f2c372d3503a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:53:10.498051 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:10.558259 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3aa21-a2de-4ed9-b170-8d550dcd2591 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:53:10.559839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3aa21-a2de-4ed9-b170-8d550dcd2591 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:10.560495 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3aa21-a2de-4ed9-b170-8d550dcd2591 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:10.560969 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3aa21-a2de-4ed9-b170-8d550dcd2591 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:10.564346 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-67b3aa21-a2de-4ed9-b170-8d550dcd2591 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:10.677362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3aa21-a2de-4ed9-b170-8d550dcd2591 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:53:10.679358 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3aa21-a2de-4ed9-b170-8d550dcd2591 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:53:10.703176 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 699/1416] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:53:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:53:11.734102 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a4d7a-6709-427c-9839-65aa378e0c31 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:11.734922 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a4d7a-6709-427c-9839-65aa378e0c31 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:11.734922 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a4d7a-6709-427c-9839-65aa378e0c31 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:11.736157 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 718/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 04:53:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:53:11.742843 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c82177-0772-43e7-9a15-773710ca345f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:11.743728 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c82177-0772-43e7-9a15-773710ca345f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:11.743728 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c82177-0772-43e7-9a15-773710ca345f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:11.840096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a5c82177-0772-43e7-9a15-773710ca345f None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:53:11.841604 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a5c82177-0772-43e7-9a15-773710ca345f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:11.877458 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:11.966968 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 700/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 04:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:53:12.127347 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a4d7a-6709-427c-9839-65aa378e0c31 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:53:12.129153 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-bd2a4d7a-6709-427c-9839-65aa378e0c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:12.318380 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a4d7a-6709-427c-9839-65aa378e0c31 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:53:12.368374 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:12.471422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be496901-ec86-4fd9-894f-c751af399d17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:53:12.472166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be496901-ec86-4fd9-894f-c751af399d17 service nova] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:12.472507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be496901-ec86-4fd9-894f-c751af399d17 service nova] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:12.472858 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be496901-ec86-4fd9-894f-c751af399d17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:12.474928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-be496901-ec86-4fd9-894f-c751af399d17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:12.592665 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be496901-ec86-4fd9-894f-c751af399d17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:53:12.594321 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be496901-ec86-4fd9-894f-c751af399d17 service nova] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:53:12.617474 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 719/1419] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:53:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:53:13.560919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f0fd3-6e71-4f84-8608-3b06edf9b69d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:13.561805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f0fd3-6e71-4f84-8608-3b06edf9b69d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:13.561805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f0fd3-6e71-4f84-8608-3b06edf9b69d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:13.563045 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 701/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 04:53:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:53:13.569306 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aec5a7-735d-46a0-901e-700b77fa04aa None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:13.570199 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aec5a7-735d-46a0-901e-700b77fa04aa None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:13.570199 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aec5a7-735d-46a0-901e-700b77fa04aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:13.639318 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:13.682283 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-33aec5a7-735d-46a0-901e-700b77fa04aa None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:53:13.684341 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-33aec5a7-735d-46a0-901e-700b77fa04aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:13.827597 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 720/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 04:53:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 04:53:13.987129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f0fd3-6e71-4f84-8608-3b06edf9b69d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:53:13.989153 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0d2f0fd3-6e71-4f84-8608-3b06edf9b69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:14.180556 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f0fd3-6e71-4f84-8608-3b06edf9b69d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:53:14.196834 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:14.328790 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33667a6a-edfa-4f76-a423-50dd226ff67c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:53:14.329340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33667a6a-edfa-4f76-a423-50dd226ff67c service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:14.329602 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33667a6a-edfa-4f76-a423-50dd226ff67c service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:14.329896 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33667a6a-edfa-4f76-a423-50dd226ff67c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:14.332219 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-33667a6a-edfa-4f76-a423-50dd226ff67c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:14.489293 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33667a6a-edfa-4f76-a423-50dd226ff67c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:53:14.492500 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33667a6a-edfa-4f76-a423-50dd226ff67c service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:53:14.520196 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 702/1422] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:53:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 04:53:15.461338 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668050d-8236-431a-bc59-6bb73821d18b None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:15.461982 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668050d-8236-431a-bc59-6bb73821d18b None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:15.461982 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668050d-8236-431a-bc59-6bb73821d18b None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:15.463589 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 721/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 04:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:53:15.470587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d2333-6769-4349-a870-d170f5379bbb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:15.471424 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d2333-6769-4349-a870-d170f5379bbb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:15.471655 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d2333-6769-4349-a870-d170f5379bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:15.496771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:15.570507 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ec2d2333-6769-4349-a870-d170f5379bbb None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:53:15.571903 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ec2d2333-6769-4349-a870-d170f5379bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:15.729200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 703/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 04:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:53:15.884472 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668050d-8236-431a-bc59-6bb73821d18b None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:53:15.886154 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-b668050d-8236-431a-bc59-6bb73821d18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:16.029050 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:16.078663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668050d-8236-431a-bc59-6bb73821d18b None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:53:16.227813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4396de50-2874-4a87-94cf-24deda509a36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:53:16.228565 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4396de50-2874-4a87-94cf-24deda509a36 service nova] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:16.228885 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4396de50-2874-4a87-94cf-24deda509a36 service nova] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:16.229192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4396de50-2874-4a87-94cf-24deda509a36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:16.231046 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-4396de50-2874-4a87-94cf-24deda509a36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:16.344255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:16.361381 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4396de50-2874-4a87-94cf-24deda509a36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:53:16.363052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4396de50-2874-4a87-94cf-24deda509a36 service nova] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:53:16.385104 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 722/1425] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:53:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:53:17.319253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b44e97-cc73-436e-9f76-97d6225f881b None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:17.319967 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b44e97-cc73-436e-9f76-97d6225f881b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:17.319967 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b44e97-cc73-436e-9f76-97d6225f881b None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:17.321281 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 704/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 04:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:53:17.327147 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc8dce-767e-4192-b756-4f4506df56e7 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:17.328095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc8dce-767e-4192-b756-4f4506df56e7 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:17.328095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc8dce-767e-4192-b756-4f4506df56e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:17.430710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-43bc8dce-767e-4192-b756-4f4506df56e7 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:53:17.432202 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-43bc8dce-767e-4192-b756-4f4506df56e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:17.571929 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 723/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 04:53:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:53:17.730814 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b44e97-cc73-436e-9f76-97d6225f881b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:53:17.732590 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-60b44e97-cc73-436e-9f76-97d6225f881b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:17.834458 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:17.909169 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b44e97-cc73-436e-9f76-97d6225f881b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:53:18.045389 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebff03b-53ba-4f26-9572-d479c035494c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:53:18.046032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebff03b-53ba-4f26-9572-d479c035494c service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:18.046364 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebff03b-53ba-4f26-9572-d479c035494c service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:18.046624 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebff03b-53ba-4f26-9572-d479c035494c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:18.048476 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bebff03b-53ba-4f26-9572-d479c035494c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:18.114681 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:18.167197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebff03b-53ba-4f26-9572-d479c035494c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:53:18.169156 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebff03b-53ba-4f26-9572-d479c035494c service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:53:18.193701 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 705/1428] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:53:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:53:19.204612 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccee0b-29eb-4115-8864-31ac1ad0282f None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:19.205263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccee0b-29eb-4115-8864-31ac1ad0282f None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:19.205263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccee0b-29eb-4115-8864-31ac1ad0282f None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:19.208120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 724/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 04:53:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:53:19.216020 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a106ec-0354-4366-b728-7f63b89d0840 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:19.216590 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a106ec-0354-4366-b728-7f63b89d0840 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:19.216709 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a106ec-0354-4366-b728-7f63b89d0840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:19.312217 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-89a106ec-0354-4366-b728-7f63b89d0840 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:53:19.313544 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-89a106ec-0354-4366-b728-7f63b89d0840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:19.426365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 706/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 04:53:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:53:19.581594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ccee0b-29eb-4115-8864-31ac1ad0282f None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:53:19.583097 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-56ccee0b-29eb-4115-8864-31ac1ad0282f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:19.641876 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:19.759965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ccee0b-29eb-4115-8864-31ac1ad0282f None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:53:19.940589 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e5265-4323-46bb-acd6-fc4f7aba96a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:53:19.941235 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e5265-4323-46bb-acd6-fc4f7aba96a0 service nova] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:19.941571 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e5265-4323-46bb-acd6-fc4f7aba96a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:19.941853 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e5265-4323-46bb-acd6-fc4f7aba96a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:19.943690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-d78e5265-4323-46bb-acd6-fc4f7aba96a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:19.952804 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:20.058442 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78e5265-4323-46bb-acd6-fc4f7aba96a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:53:20.060490 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78e5265-4323-46bb-acd6-fc4f7aba96a0 service nova] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:53:20.084850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 725/1431] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:53:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:53:20.918905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18e3f3-cce8-472b-b691-547aaea67f87 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:20.919540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18e3f3-cce8-472b-b691-547aaea67f87 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:20.919840 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18e3f3-cce8-472b-b691-547aaea67f87 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:20.921537 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 707/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 04:53:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:53:20.927575 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa4a01-9753-49b1-8637-840355dd7c4e None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:20.928172 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa4a01-9753-49b1-8637-840355dd7c4e None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:20.928172 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa4a01-9753-49b1-8637-840355dd7c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:21.022073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-66fa4a01-9753-49b1-8637-840355dd7c4e None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:53:21.023235 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-66fa4a01-9753-49b1-8637-840355dd7c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:21.106408 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:21.145155 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 726/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 04:53:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:53:21.301112 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb18e3f3-cce8-472b-b691-547aaea67f87 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:53:21.302633 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fb18e3f3-cce8-472b-b691-547aaea67f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:21.323613 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:21.486691 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb18e3f3-cce8-472b-b691-547aaea67f87 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:53:21.620096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:21.663762 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e13876-eb2f-4250-a92e-0ffc959faf2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:53:21.664612 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e13876-eb2f-4250-a92e-0ffc959faf2c service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:21.664984 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e13876-eb2f-4250-a92e-0ffc959faf2c service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:21.665343 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e13876-eb2f-4250-a92e-0ffc959faf2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:21.667294 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-04e13876-eb2f-4250-a92e-0ffc959faf2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:21.774294 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e13876-eb2f-4250-a92e-0ffc959faf2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:53:21.776014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e13876-eb2f-4250-a92e-0ffc959faf2c service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:53:21.806723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 708/1434] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:53:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:53:22.696037 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6a873-1a37-4950-8e89-544727b0722b None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:22.696660 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6a873-1a37-4950-8e89-544727b0722b None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:22.696974 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6a873-1a37-4950-8e89-544727b0722b None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:22.699680 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 727/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 04:53:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:53:22.709194 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad38f6d-0e78-43dd-b17b-ffc81d12e862 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:22.709987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad38f6d-0e78-43dd-b17b-ffc81d12e862 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:22.710129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad38f6d-0e78-43dd-b17b-ffc81d12e862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:22.814761 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1ad38f6d-0e78-43dd-b17b-ffc81d12e862 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:53:22.816148 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1ad38f6d-0e78-43dd-b17b-ffc81d12e862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:22.866713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:22.977185 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 709/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 04:53:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:53:23.135358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6a873-1a37-4950-8e89-544727b0722b None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:53:23.137025 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-4bb6a873-1a37-4950-8e89-544727b0722b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:23.315134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6a873-1a37-4950-8e89-544727b0722b None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:53:23.380151 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:23.453994 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda25d9-6653-4c5c-8240-d14102e2f8c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:53:23.454752 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda25d9-6653-4c5c-8240-d14102e2f8c7 service nova] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:23.455103 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda25d9-6653-4c5c-8240-d14102e2f8c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:23.455422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda25d9-6653-4c5c-8240-d14102e2f8c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:23.457667 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-cbda25d9-6653-4c5c-8240-d14102e2f8c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:23.563476 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda25d9-6653-4c5c-8240-d14102e2f8c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:53:23.565605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda25d9-6653-4c5c-8240-d14102e2f8c7 service nova] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:53:23.589815 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 728/1437] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:53:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:53:24.465016 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1ed4c-2178-41db-b273-84e667e3c34e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:24.465630 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1ed4c-2178-41db-b273-84e667e3c34e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:24.465630 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1ed4c-2178-41db-b273-84e667e3c34e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:24.466948 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 710/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 04:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:53:24.472624 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd9e83-be7a-4510-b679-54d4d40eed81 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:24.473183 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd9e83-be7a-4510-b679-54d4d40eed81 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:24.473183 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd9e83-be7a-4510-b679-54d4d40eed81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:24.564534 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-7edd9e83-be7a-4510-b679-54d4d40eed81 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:53:24.565809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-7edd9e83-be7a-4510-b679-54d4d40eed81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:24.568265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:24.679063 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 729/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 04:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:53:24.833619 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1ed4c-2178-41db-b273-84e667e3c34e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:53:24.835330 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-24f1ed4c-2178-41db-b273-84e667e3c34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:25.014150 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1ed4c-2178-41db-b273-84e667e3c34e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:53:25.047330 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:25.153748 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388949d7-c53a-4a68-96ed-23775c942674 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:53:25.154572 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388949d7-c53a-4a68-96ed-23775c942674 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:25.154856 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388949d7-c53a-4a68-96ed-23775c942674 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:25.155275 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388949d7-c53a-4a68-96ed-23775c942674 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:25.157106 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-388949d7-c53a-4a68-96ed-23775c942674 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:25.267820 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388949d7-c53a-4a68-96ed-23775c942674 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:53:25.269369 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388949d7-c53a-4a68-96ed-23775c942674 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:53:25.291797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 711/1440] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:53:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:53:27.316988 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c03b0-12cb-4088-a144-18bc39a1c1be None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:27.317891 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c03b0-12cb-4088-a144-18bc39a1c1be None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:27.317891 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c03b0-12cb-4088-a144-18bc39a1c1be None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:27.319583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 730/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 04:53:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:53:27.326266 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5021d5-9e13-470a-a3af-af3654f7f32e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:27.326916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5021d5-9e13-470a-a3af-af3654f7f32e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:27.326916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5021d5-9e13-470a-a3af-af3654f7f32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:27.396390 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:27.437381 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5c5021d5-9e13-470a-a3af-af3654f7f32e None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:53:27.439115 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5c5021d5-9e13-470a-a3af-af3654f7f32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:27.591243 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 712/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 04:53:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:53:27.636495 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c03b0-12cb-4088-a144-18bc39a1c1be None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:53:27.638099 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-b94c03b0-12cb-4088-a144-18bc39a1c1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:27.773964 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c03b0-12cb-4088-a144-18bc39a1c1be None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:53:27.838192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:27.900815 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5688cf9f-59db-45df-94b4-03b2d383aec4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:53:27.901454 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688cf9f-59db-45df-94b4-03b2d383aec4 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:27.901737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688cf9f-59db-45df-94b4-03b2d383aec4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:27.902013 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688cf9f-59db-45df-94b4-03b2d383aec4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:27.903825 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-5688cf9f-59db-45df-94b4-03b2d383aec4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:28.035424 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5688cf9f-59db-45df-94b4-03b2d383aec4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:53:28.037118 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5688cf9f-59db-45df-94b4-03b2d383aec4 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:53:28.058409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 731/1443] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:53:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:53:30.394972 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0874f-8469-411d-9ef0-e22bce837a66 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:30.395620 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0874f-8469-411d-9ef0-e22bce837a66 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:30.395750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0874f-8469-411d-9ef0-e22bce837a66 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:30.397453 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 713/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 04:53:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:53:30.404124 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038302dc-77b1-460b-aa17-e590c61b4c65 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:30.404792 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038302dc-77b1-460b-aa17-e590c61b4c65 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:30.404792 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038302dc-77b1-460b-aa17-e590c61b4c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:30.487078 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-038302dc-77b1-460b-aa17-e590c61b4c65 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:53:30.488357 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-038302dc-77b1-460b-aa17-e590c61b4c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:30.503573 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:30.607250 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 732/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 04:53:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 04:53:30.657717 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc0874f-8469-411d-9ef0-e22bce837a66 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:53:30.660337 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8dc0874f-8469-411d-9ef0-e22bce837a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:30.825509 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc0874f-8469-411d-9ef0-e22bce837a66 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:53:30.841249 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:30.921609 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f77418-e403-4e21-bfd0-273c4cd14628 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:53:30.922292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f77418-e403-4e21-bfd0-273c4cd14628 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:30.922576 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f77418-e403-4e21-bfd0-273c4cd14628 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:30.922852 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f77418-e403-4e21-bfd0-273c4cd14628 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:30.924644 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f0f77418-e403-4e21-bfd0-273c4cd14628 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:31.067269 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f77418-e403-4e21-bfd0-273c4cd14628 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:53:31.070766 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f77418-e403-4e21-bfd0-273c4cd14628 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:53:31.109347 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 714/1446] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:53:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:53:33.925473 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390822d-43e9-4885-b894-45c40bb0d9e3 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:33.926182 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390822d-43e9-4885-b894-45c40bb0d9e3 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:33.926182 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390822d-43e9-4885-b894-45c40bb0d9e3 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:33.927620 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 733/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 04:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:53:33.934927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5970b1-445f-48bc-9162-73a90927ea15 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:33.935536 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5970b1-445f-48bc-9162-73a90927ea15 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:33.935937 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5970b1-445f-48bc-9162-73a90927ea15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:34.035931 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:34.058854 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ba5970b1-445f-48bc-9162-73a90927ea15 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:53:34.060750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ba5970b1-445f-48bc-9162-73a90927ea15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:34.187643 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 715/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 04:53:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:53:34.229157 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390822d-43e9-4885-b894-45c40bb0d9e3 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:53:34.230848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-5390822d-43e9-4885-b894-45c40bb0d9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:34.370567 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390822d-43e9-4885-b894-45c40bb0d9e3 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:53:34.420460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:34.460133 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230b44a-4b9b-49d3-a4f4-7a31a7f696dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:53:34.460674 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230b44a-4b9b-49d3-a4f4-7a31a7f696dd service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:34.460958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230b44a-4b9b-49d3-a4f4-7a31a7f696dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:34.461231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230b44a-4b9b-49d3-a4f4-7a31a7f696dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:34.462941 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-d230b44a-4b9b-49d3-a4f4-7a31a7f696dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:34.590756 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230b44a-4b9b-49d3-a4f4-7a31a7f696dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:53:34.592290 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230b44a-4b9b-49d3-a4f4-7a31a7f696dd service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:53:34.614885 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 734/1449] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:53:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:53:36.794394 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8db9b-d65e-451b-b334-d49848a47aca None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:36.794998 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8db9b-d65e-451b-b334-d49848a47aca None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:36.794998 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8db9b-d65e-451b-b334-d49848a47aca None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:36.796385 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 716/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 04:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:53:36.802495 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c840ffc-df90-497d-b6a7-6d430e75f46b None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:36.803021 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c840ffc-df90-497d-b6a7-6d430e75f46b None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:36.803115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c840ffc-df90-497d-b6a7-6d430e75f46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:36.881630 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-2c840ffc-df90-497d-b6a7-6d430e75f46b None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:53:36.882930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-2c840ffc-df90-497d-b6a7-6d430e75f46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:36.906716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:37.011110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 735/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 04:53:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:53:37.053671 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b8db9b-d65e-451b-b334-d49848a47aca None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:53:37.055286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d9b8db9b-d65e-451b-b334-d49848a47aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:37.179321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b8db9b-d65e-451b-b334-d49848a47aca None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:53:37.203191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:37.270602 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f33296-2dc3-414c-828e-699d561bd1cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:53:37.271240 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f33296-2dc3-414c-828e-699d561bd1cc service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:37.271521 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f33296-2dc3-414c-828e-699d561bd1cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:37.271827 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f33296-2dc3-414c-828e-699d561bd1cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:37.273513 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b5f33296-2dc3-414c-828e-699d561bd1cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:37.402170 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f33296-2dc3-414c-828e-699d561bd1cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:53:37.403665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f33296-2dc3-414c-828e-699d561bd1cc service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:53:37.424396 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 717/1452] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:53:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:53:39.650391 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049ede0-a6b4-49cd-ab02-6f015a038f56 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:39.650986 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049ede0-a6b4-49cd-ab02-6f015a038f56 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:39.650986 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049ede0-a6b4-49cd-ab02-6f015a038f56 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:39.652772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 736/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 04:53:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:53:39.659109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d35d54-d30d-4dc1-a9d2-a07acdd61ccf None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:39.659742 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d35d54-d30d-4dc1-a9d2-a07acdd61ccf None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:39.659742 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d35d54-d30d-4dc1-a9d2-a07acdd61ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:39.750158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:39.754434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-84d35d54-d30d-4dc1-a9d2-a07acdd61ccf None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:53:39.756906 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-84d35d54-d30d-4dc1-a9d2-a07acdd61ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:39.861118 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 718/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 04:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:53:39.898955 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049ede0-a6b4-49cd-ab02-6f015a038f56 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:53:39.900902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-2049ede0-a6b4-49cd-ab02-6f015a038f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:40.022999 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049ede0-a6b4-49cd-ab02-6f015a038f56 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:53:40.083199 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:40.114076 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b27f7-31ea-4871-ac39-aa63d2a5c8c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:53:40.114735 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b27f7-31ea-4871-ac39-aa63d2a5c8c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:40.115036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b27f7-31ea-4871-ac39-aa63d2a5c8c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:40.115302 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b27f7-31ea-4871-ac39-aa63d2a5c8c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:40.117090 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a49b27f7-31ea-4871-ac39-aa63d2a5c8c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:40.250444 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b27f7-31ea-4871-ac39-aa63d2a5c8c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:53:40.251892 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b27f7-31ea-4871-ac39-aa63d2a5c8c6 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:53:40.274084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 737/1455] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:53:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:53:42.471386 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a5261-d1b3-4322-919d-ccf93782f2d2 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:42.471934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a5261-d1b3-4322-919d-ccf93782f2d2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:42.471934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a5261-d1b3-4322-919d-ccf93782f2d2 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:42.474126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 719/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 04:53:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:53:42.482767 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41129747-5288-478e-88d4-e72323c57558 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:42.483209 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41129747-5288-478e-88d4-e72323c57558 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:42.483347 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41129747-5288-478e-88d4-e72323c57558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:42.557118 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-41129747-5288-478e-88d4-e72323c57558 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:53:42.558353 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-41129747-5288-478e-88d4-e72323c57558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:42.589714 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:42.668632 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 738/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 04:53:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:53:42.713714 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a5261-d1b3-4322-919d-ccf93782f2d2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:53:42.716036 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-267a5261-d1b3-4322-919d-ccf93782f2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:42.837182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a5261-d1b3-4322-919d-ccf93782f2d2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:53:42.866230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:42.917393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5868183-10a2-486d-98bf-66a585749940 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:53:42.918034 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5868183-10a2-486d-98bf-66a585749940 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:42.918320 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5868183-10a2-486d-98bf-66a585749940 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:42.918598 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5868183-10a2-486d-98bf-66a585749940 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:42.920336 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f5868183-10a2-486d-98bf-66a585749940 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:43.070471 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5868183-10a2-486d-98bf-66a585749940 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:53:43.074211 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5868183-10a2-486d-98bf-66a585749940 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:53:43.110929 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 720/1458] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:53:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 04:53:45.237542 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d310c35-5e97-4c69-b05b-968b58f1c801 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:45.238313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d310c35-5e97-4c69-b05b-968b58f1c801 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:45.238313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d310c35-5e97-4c69-b05b-968b58f1c801 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:45.239617 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 739/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 04:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:53:45.245808 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c301885-a2d4-4cb1-b1a6-8e176b8d615f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:45.246473 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c301885-a2d4-4cb1-b1a6-8e176b8d615f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:45.246473 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c301885-a2d4-4cb1-b1a6-8e176b8d615f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:45.326611 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7c301885-a2d4-4cb1-b1a6-8e176b8d615f None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:53:45.327904 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7c301885-a2d4-4cb1-b1a6-8e176b8d615f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:45.340844 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:45.435341 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 721/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 04:53:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:53:45.473519 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d310c35-5e97-4c69-b05b-968b58f1c801 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:53:45.475132 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-9d310c35-5e97-4c69-b05b-968b58f1c801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:45.591105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d310c35-5e97-4c69-b05b-968b58f1c801 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:53:45.652878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:45.673067 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32fb6e4-875d-4c7e-9736-b51a20eee5e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:53:45.673686 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fb6e4-875d-4c7e-9736-b51a20eee5e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:45.673961 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fb6e4-875d-4c7e-9736-b51a20eee5e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:45.674231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fb6e4-875d-4c7e-9736-b51a20eee5e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:45.675951 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-b32fb6e4-875d-4c7e-9736-b51a20eee5e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:45.803177 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32fb6e4-875d-4c7e-9736-b51a20eee5e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:53:45.804799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32fb6e4-875d-4c7e-9736-b51a20eee5e0 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:53:45.827290 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 740/1461] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:53:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:53:47.904051 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc578aa-7cae-4ac4-b961-240c185343c2 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:47.904691 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc578aa-7cae-4ac4-b961-240c185343c2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:47.904691 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc578aa-7cae-4ac4-b961-240c185343c2 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:47.905999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 722/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 04:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:53:47.912220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74672f86-bca1-4f9b-961b-02dc2f79fcfd None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:47.912860 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74672f86-bca1-4f9b-961b-02dc2f79fcfd None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:47.912860 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74672f86-bca1-4f9b-961b-02dc2f79fcfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:47.989267 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-74672f86-bca1-4f9b-961b-02dc2f79fcfd None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:53:47.990552 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-74672f86-bca1-4f9b-961b-02dc2f79fcfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:48.036905 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:48.107947 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 741/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 04:53:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:53:48.151461 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc578aa-7cae-4ac4-b961-240c185343c2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:53:48.153056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9cc578aa-7cae-4ac4-b961-240c185343c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:48.275127 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc578aa-7cae-4ac4-b961-240c185343c2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:53:48.314431 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:48.356988 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7c6d3-19d4-4a71-9249-029300c2e11e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:53:48.356988 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7c6d3-19d4-4a71-9249-029300c2e11e service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:48.357301 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7c6d3-19d4-4a71-9249-029300c2e11e service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:48.357579 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7c6d3-19d4-4a71-9249-029300c2e11e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:48.359288 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a7c7c6d3-19d4-4a71-9249-029300c2e11e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:48.492200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c7c6d3-19d4-4a71-9249-029300c2e11e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:53:48.493605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c7c6d3-19d4-4a71-9249-029300c2e11e service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:53:48.513379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 723/1464] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:53:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:53:50.641085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb371ad9-cf5b-41f0-9b3c-f2f3ec41b87e None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:50.641671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb371ad9-cf5b-41f0-9b3c-f2f3ec41b87e None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:50.641671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb371ad9-cf5b-41f0-9b3c-f2f3ec41b87e None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:50.644006 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 742/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 04:53:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:53:50.650368 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cbc2d-4a63-4ba1-9e73-13807cead895 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:50.651002 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cbc2d-4a63-4ba1-9e73-13807cead895 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:50.651002 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cbc2d-4a63-4ba1-9e73-13807cead895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:50.733487 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5a6cbc2d-4a63-4ba1-9e73-13807cead895 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:53:50.734752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5a6cbc2d-4a63-4ba1-9e73-13807cead895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:50.755473 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:50.852606 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 724/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 04:53:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 04:53:50.891779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb371ad9-cf5b-41f0-9b3c-f2f3ec41b87e None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:53:50.893280 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-eb371ad9-cf5b-41f0-9b3c-f2f3ec41b87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:51.008966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb371ad9-cf5b-41f0-9b3c-f2f3ec41b87e None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:53:51.094611 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:51.104487 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd5f62-980b-4851-9fd8-7b8fd4de081a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:53:51.105431 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd5f62-980b-4851-9fd8-7b8fd4de081a service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:51.105886 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd5f62-980b-4851-9fd8-7b8fd4de081a service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:51.106387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd5f62-980b-4851-9fd8-7b8fd4de081a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:51.109170 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-3fcd5f62-980b-4851-9fd8-7b8fd4de081a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:51.237932 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd5f62-980b-4851-9fd8-7b8fd4de081a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:53:51.239733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd5f62-980b-4851-9fd8-7b8fd4de081a service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:53:51.265631 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 743/1467] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:53:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:53:53.395086 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f306f-80fe-4a8f-b428-3bc702c1f3f8 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:53.395694 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f306f-80fe-4a8f-b428-3bc702c1f3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:53.395840 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f306f-80fe-4a8f-b428-3bc702c1f3f8 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:53.397308 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 725/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 04:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:53:53.403456 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d137e32-aa48-455e-8cd0-b3b1324743cc None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:53.404009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d137e32-aa48-455e-8cd0-b3b1324743cc None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:53.404127 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d137e32-aa48-455e-8cd0-b3b1324743cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:53.472736 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-9d137e32-aa48-455e-8cd0-b3b1324743cc None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:53:53.474052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-9d137e32-aa48-455e-8cd0-b3b1324743cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:53.531645 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:53.587432 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 744/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 04:53:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:53:53.625656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f306f-80fe-4a8f-b428-3bc702c1f3f8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:53:53.627272 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3b0f306f-80fe-4a8f-b428-3bc702c1f3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:53.752980 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f306f-80fe-4a8f-b428-3bc702c1f3f8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:53:53.808648 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:53:53.844904 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa1c1c-30d8-4430-adf5-a45e282256e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:53:53.845580 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa1c1c-30d8-4430-adf5-a45e282256e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:53:53.845893 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa1c1c-30d8-4430-adf5-a45e282256e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:53:53.846176 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa1c1c-30d8-4430-adf5-a45e282256e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:53:53.848011 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e6fa1c1c-30d8-4430-adf5-a45e282256e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:53:53.983905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa1c1c-30d8-4430-adf5-a45e282256e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:53:53.985413 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa1c1c-30d8-4430-adf5-a45e282256e0 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:53:54.006271 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 726/1470] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:53:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:54:03.584339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=22 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:54:04.003462 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=66 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:54:14.120075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05929319-a650-477c-8b8c-deb78df9031d None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:54:14.120711 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05929319-a650-477c-8b8c-deb78df9031d None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:54:14.120711 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05929319-a650-477c-8b8c-deb78df9031d None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:54:14.122175 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 745/1471] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:54:14.127839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0a448-5e18-40da-8ca2-70d1ade700e6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:54:14.128422 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0a448-5e18-40da-8ca2-70d1ade700e6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:54:14.128422 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0a448-5e18-40da-8ca2-70d1ade700e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:54:14.202680 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9bb0a448-5e18-40da-8ca2-70d1ade700e6 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:54:14.203975 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9bb0a448-5e18-40da-8ca2-70d1ade700e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:54:14.313199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 727/1472] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:54:14.409032 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d906f-da66-46b9-8408-fcf0fcacddc5 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:54:14.409324 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d906f-da66-46b9-8408-fcf0fcacddc5 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:54:14.409668 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d906f-da66-46b9-8408-fcf0fcacddc5 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:54:14.411164 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 746/1473] 10.222.0.17 () {54 vars in 966 bytes} [Tue Jul 26 04:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:54:14.416615 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e5f0e-a239-4062-92ef-9f50a4a3a0e1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:54:14.416897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e5f0e-a239-4062-92ef-9f50a4a3a0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:54:14.417170 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e5f0e-a239-4062-92ef-9f50a4a3a0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:54:14.507742 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-bc1e5f0e-a239-4062-92ef-9f50a4a3a0e1 None None] MFA Rules not processed for user `6946a788d2ef49ff873ca8e81990527e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:54:14.508988 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bc1e5f0e-a239-4062-92ef-9f50a4a3a0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:54:14.531368 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:54:14.598450 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 728/1474] 10.222.0.17 () {58 vars in 1063 bytes} [Tue Jul 26 04:54:14 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 04:54:14.603428 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d906f-da66-46b9-8408-fcf0fcacddc5 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:54:14.605054 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-150d906f-da66-46b9-8408-fcf0fcacddc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:54:14.719108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d906f-da66-46b9-8408-fcf0fcacddc5 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:54:14.800598 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532f47d-9d8b-47fb-9da5-ce5278e09926 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6946a788d2ef49ff873ca8e81990527e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:54:14.801292 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532f47d-9d8b-47fb-9da5-ce5278e09926 service glance] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:54:14.801601 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532f47d-9d8b-47fb-9da5-ce5278e09926 service glance] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:54:14.801970 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532f47d-9d8b-47fb-9da5-ce5278e09926 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:54:14.804760 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c532f47d-9d8b-47fb-9da5-ce5278e09926 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:54:14.870675 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:54:14.933806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c532f47d-9d8b-47fb-9da5-ce5278e09926 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:54:14.935307 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c532f47d-9d8b-47fb-9da5-ce5278e09926 service glance] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:54:14.954629 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 747/1475] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:54:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 04:54:17.078545 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815e1b5-3251-4222-a52e-f6c9f48888d0 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:54:17.079454 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815e1b5-3251-4222-a52e-f6c9f48888d0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:54:17.079454 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815e1b5-3251-4222-a52e-f6c9f48888d0 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:54:17.081643 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 729/1476] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:54:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:54:17.087662 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749b741-0bec-4c52-9a75-b7a17d6a1b54 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:54:17.088113 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749b741-0bec-4c52-9a75-b7a17d6a1b54 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:54:17.088221 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749b741-0bec-4c52-9a75-b7a17d6a1b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:54:17.174458 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-8749b741-0bec-4c52-9a75-b7a17d6a1b54 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:54:17.175653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-8749b741-0bec-4c52-9a75-b7a17d6a1b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:54:17.292748 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 748/1477] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:54:17.316642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f815e1b5-3251-4222-a52e-f6c9f48888d0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:54:17.318711 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f815e1b5-3251-4222-a52e-f6c9f48888d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:54:17.462571 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f815e1b5-3251-4222-a52e-f6c9f48888d0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:54:17.514846 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:54:17.575449 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb21f5-16b8-480a-a9db-b23648ef40aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:54:17.576121 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb21f5-16b8-480a-a9db-b23648ef40aa admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:54:17.576400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb21f5-16b8-480a-a9db-b23648ef40aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:54:17.576669 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb21f5-16b8-480a-a9db-b23648ef40aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:54:17.577371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb21f5-16b8-480a-a9db-b23648ef40aa admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:54:17.579209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-c3cb21f5-16b8-480a-a9db-b23648ef40aa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 04:54:17.579209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jul 26 04:54:17.579209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: Jul 26 04:54:17.580400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb21f5-16b8-480a-a9db-b23648ef40aa admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:54:17.601214 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 730/1478] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jul 26 04:54:17 2022] GET /identity/v3/projects => generated 2553 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 04:54:19.715679 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df890ad-a7f1-484e-9e9f-f4ccb668fb72 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:54:19.716280 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df890ad-a7f1-484e-9e9f-f4ccb668fb72 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:54:19.716280 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df890ad-a7f1-484e-9e9f-f4ccb668fb72 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:54:19.717592 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 749/1479] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:54:19.724049 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e9accc-1eb6-410a-ba84-68303b7d526f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:54:19.724583 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e9accc-1eb6-410a-ba84-68303b7d526f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:54:19.724748 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e9accc-1eb6-410a-ba84-68303b7d526f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:54:19.802690 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-28e9accc-1eb6-410a-ba84-68303b7d526f None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:54:19.804370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-28e9accc-1eb6-410a-ba84-68303b7d526f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:54:19.911744 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 731/1480] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:54:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:54:19.955586 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df890ad-a7f1-484e-9e9f-f4ccb668fb72 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:54:19.957198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-9df890ad-a7f1-484e-9e9f-f4ccb668fb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:54:19.968263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:54:20.119573 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df890ad-a7f1-484e-9e9f-f4ccb668fb72 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:54:20.202035 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:54:20.242804 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59fee4-d956-4d36-ba4b-d3f96bd3c9f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:54:20.243332 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59fee4-d956-4d36-ba4b-d3f96bd3c9f5 service nova] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:54:20.243649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59fee4-d956-4d36-ba4b-d3f96bd3c9f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:54:20.243926 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59fee4-d956-4d36-ba4b-d3f96bd3c9f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:54:20.245660 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-1f59fee4-d956-4d36-ba4b-d3f96bd3c9f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:54:20.363816 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f59fee4-d956-4d36-ba4b-d3f96bd3c9f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:54:20.365481 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f59fee4-d956-4d36-ba4b-d3f96bd3c9f5 service nova] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:54:20.388130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 750/1481] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:54:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:54:22.617830 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa6da9-bd22-4d7f-90b0-caa4dd89fe10 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:54:22.618408 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa6da9-bd22-4d7f-90b0-caa4dd89fe10 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:54:22.618408 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa6da9-bd22-4d7f-90b0-caa4dd89fe10 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:54:22.619829 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 732/1482] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:54:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:54:22.626157 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe370a38-c715-487e-a430-968a1fd1b8f1 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:54:22.626715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe370a38-c715-487e-a430-968a1fd1b8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:54:22.626932 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe370a38-c715-487e-a430-968a1fd1b8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:54:22.727530 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-fe370a38-c715-487e-a430-968a1fd1b8f1 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:54:22.729698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-fe370a38-c715-487e-a430-968a1fd1b8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:54:22.785312 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:54:22.841737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 751/1483] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:54:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 04:54:22.882867 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa6da9-bd22-4d7f-90b0-caa4dd89fe10 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:54:22.884656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d8aa6da9-bd22-4d7f-90b0-caa4dd89fe10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:54:22.912618 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:54:23.016016 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa6da9-bd22-4d7f-90b0-caa4dd89fe10 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:54:23.096934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400670b-56b3-4b52-bd72-dc85d9478f49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:54:23.097622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400670b-56b3-4b52-bd72-dc85d9478f49 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:54:23.097908 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400670b-56b3-4b52-bd72-dc85d9478f49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:54:23.098214 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400670b-56b3-4b52-bd72-dc85d9478f49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:54:23.100033 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f400670b-56b3-4b52-bd72-dc85d9478f49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:54:23.182144 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:54:23.223276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400670b-56b3-4b52-bd72-dc85d9478f49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:54:23.224906 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400670b-56b3-4b52-bd72-dc85d9478f49 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:54:23.245806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 733/1484] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:54:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:54:25.370584 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405307d3-1bc9-4468-ba4a-766bbe609a2c None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:54:25.371263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405307d3-1bc9-4468-ba4a-766bbe609a2c None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:54:25.371371 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405307d3-1bc9-4468-ba4a-766bbe609a2c None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:54:25.372945 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 752/1485] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:54:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:54:25.379179 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ee943-fb11-492f-8db7-45ffd1351caa None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:54:25.379781 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ee943-fb11-492f-8db7-45ffd1351caa None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:54:25.379781 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ee943-fb11-492f-8db7-45ffd1351caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:54:25.455565 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-813ee943-fb11-492f-8db7-45ffd1351caa None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:54:25.456832 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-813ee943-fb11-492f-8db7-45ffd1351caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:54:25.580692 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 734/1486] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:54:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:54:25.627723 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405307d3-1bc9-4468-ba4a-766bbe609a2c None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:54:25.629723 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-405307d3-1bc9-4468-ba4a-766bbe609a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:54:25.754419 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405307d3-1bc9-4468-ba4a-766bbe609a2c None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:54:25.836670 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:54:25.838948 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e428a8ad-6ebd-4965-9723-d6b100bd4bad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:54:25.839665 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428a8ad-6ebd-4965-9723-d6b100bd4bad service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:54:25.839959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428a8ad-6ebd-4965-9723-d6b100bd4bad service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:54:25.840240 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428a8ad-6ebd-4965-9723-d6b100bd4bad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:54:25.842806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-e428a8ad-6ebd-4965-9723-d6b100bd4bad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:54:25.965705 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e428a8ad-6ebd-4965-9723-d6b100bd4bad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:54:25.967074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e428a8ad-6ebd-4965-9723-d6b100bd4bad service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:54:25.987839 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 753/1487] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:54:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:54:28.312619 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51d1be-118e-4828-ae3e-91983ecc1b64 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:54:28.313244 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51d1be-118e-4828-ae3e-91983ecc1b64 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:54:28.313244 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51d1be-118e-4828-ae3e-91983ecc1b64 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:54:28.314732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 735/1488] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:54:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:54:28.320590 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7558c1a-4cbc-4cd2-b3f0-c1dfcc0a432b None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:54:28.321100 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7558c1a-4cbc-4cd2-b3f0-c1dfcc0a432b None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:54:28.321243 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7558c1a-4cbc-4cd2-b3f0-c1dfcc0a432b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:54:28.398170 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-c7558c1a-4cbc-4cd2-b3f0-c1dfcc0a432b None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:54:28.399421 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c7558c1a-4cbc-4cd2-b3f0-c1dfcc0a432b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:54:28.456903 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:54:28.539604 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 754/1489] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:54:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:54:28.580599 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51d1be-118e-4828-ae3e-91983ecc1b64 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:54:28.582209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-be51d1be-118e-4828-ae3e-91983ecc1b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:54:28.626003 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:54:28.706333 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51d1be-118e-4828-ae3e-91983ecc1b64 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:54:28.787795 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdea025-ca5b-4759-9ee7-9bc944576297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:54:28.788435 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdea025-ca5b-4759-9ee7-9bc944576297 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:54:28.788719 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdea025-ca5b-4759-9ee7-9bc944576297 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:54:28.788992 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdea025-ca5b-4759-9ee7-9bc944576297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:54:28.790650 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8bdea025-ca5b-4759-9ee7-9bc944576297 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:54:28.884985 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:54:28.918485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdea025-ca5b-4759-9ee7-9bc944576297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:54:28.920066 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdea025-ca5b-4759-9ee7-9bc944576297 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:54:28.940432 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 736/1490] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:54:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:54:31.312888 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961fc43-8fb2-477a-ae50-0a28184da93b None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:54:31.313528 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961fc43-8fb2-477a-ae50-0a28184da93b None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:54:31.313873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961fc43-8fb2-477a-ae50-0a28184da93b None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:54:31.316607 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 755/1491] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jul 26 04:54:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:54:31.322733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bcf24-46cf-4947-a2dd-a745455f7250 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:54:31.323356 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bcf24-46cf-4947-a2dd-a745455f7250 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:54:31.323356 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bcf24-46cf-4947-a2dd-a745455f7250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:54:31.402487 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1b6bcf24-46cf-4947-a2dd-a745455f7250 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:54:31.403832 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1b6bcf24-46cf-4947-a2dd-a745455f7250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:54:31.517972 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 737/1492] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jul 26 04:54:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:54:31.559687 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961fc43-8fb2-477a-ae50-0a28184da93b None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:54:31.561491 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-1961fc43-8fb2-477a-ae50-0a28184da93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:54:31.704349 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961fc43-8fb2-477a-ae50-0a28184da93b None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:54:31.809896 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bcc202-f066-4dcd-8034-ccf028c43ceb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:54:31.810934 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcc202-f066-4dcd-8034-ccf028c43ceb service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:54:31.811427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcc202-f066-4dcd-8034-ccf028c43ceb service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:54:31.811980 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcc202-f066-4dcd-8034-ccf028c43ceb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:54:31.815596 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-e9bcc202-f066-4dcd-8034-ccf028c43ceb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:54:31.827330 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:54:31.963709 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bcc202-f066-4dcd-8034-ccf028c43ceb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:54:31.965114 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bcc202-f066-4dcd-8034-ccf028c43ceb service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:54:31.986221 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 756/1493] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:54:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:54:41.515743 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=82 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:54:41.983770 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=50 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:56:10.410331 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058784d4-b8a9-4c87-98c1-f0d93fc6ad7b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:56:10.410852 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058784d4-b8a9-4c87-98c1-f0d93fc6ad7b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:56:10.410852 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058784d4-b8a9-4c87-98c1-f0d93fc6ad7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:56:10.493812 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-058784d4-b8a9-4c87-98c1-f0d93fc6ad7b None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:56:10.495070 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-058784d4-b8a9-4c87-98c1-f0d93fc6ad7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:56:10.597541 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 738/1494] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:56:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:56:10.606152 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0b09e-655f-4dc7-9d8a-ddd8497b0fe4 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:56:10.606751 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0b09e-655f-4dc7-9d8a-ddd8497b0fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:56:10.606751 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0b09e-655f-4dc7-9d8a-ddd8497b0fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:56:10.684636 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-e8f0b09e-655f-4dc7-9d8a-ddd8497b0fe4 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:56:10.685763 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-e8f0b09e-655f-4dc7-9d8a-ddd8497b0fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:56:10.783514 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 757/1495] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:56:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:56:10.788399 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058784d4-b8a9-4c87-98c1-f0d93fc6ad7b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:56:10.789605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-058784d4-b8a9-4c87-98c1-f0d93fc6ad7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:56:10.801174 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058784d4-b8a9-4c87-98c1-f0d93fc6ad7b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:56:10.871891 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:56:10.908554 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff4d5c-fa9f-47fb-8072-b56184ae5fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:56:10.910092 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff4d5c-fa9f-47fb-8072-b56184ae5fc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:56:10.910664 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff4d5c-fa9f-47fb-8072-b56184ae5fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:56:10.911362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff4d5c-fa9f-47fb-8072-b56184ae5fc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:56:10.913192 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff4d5c-fa9f-47fb-8072-b56184ae5fc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:56:10.913905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff4d5c-fa9f-47fb-8072-b56184ae5fc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:56:10.916545 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-87ff4d5c-fa9f-47fb-8072-b56184ae5fc6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 04:56:10.916545 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jul 26 04:56:10.916545 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: Jul 26 04:56:10.918871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff4d5c-fa9f-47fb-8072-b56184ae5fc6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:56:10.943848 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 739/1496] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 04:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:56:10.952278 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0b09e-655f-4dc7-9d8a-ddd8497b0fe4 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:56:10.962847 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0b09e-655f-4dc7-9d8a-ddd8497b0fe4 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:56:11.036622 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:56:11.074268 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5a37c-5fbf-430c-981b-b7bac19a29df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:56:11.074898 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5a37c-5fbf-430c-981b-b7bac19a29df admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:56:11.075182 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5a37c-5fbf-430c-981b-b7bac19a29df admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:56:11.075486 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5a37c-5fbf-430c-981b-b7bac19a29df admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:56:11.076189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed5a37c-5fbf-430c-981b-b7bac19a29df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:56:11.077954 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: WARNING py.warnings [None req-1ed5a37c-5fbf-430c-981b-b7bac19a29df admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 04:56:11.077954 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: warnings.warn(msg) Jul 26 04:56:11.077954 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: Jul 26 04:56:11.079179 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed5a37c-5fbf-430c-981b-b7bac19a29df admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:56:11.141840 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 758/1497] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 04:56:10 2022] POST /identity/v3/projects => generated 377 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:56:11.146454 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff4d5c-fa9f-47fb-8072-b56184ae5fc6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:56:11.148169 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-87ff4d5c-fa9f-47fb-8072-b56184ae5fc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:56:11.184170 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff4d5c-fa9f-47fb-8072-b56184ae5fc6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:56:11.291828 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f4ded-407e-4b55-8808-14f38abf62cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:56:11.292361 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f4ded-407e-4b55-8808-14f38abf62cc admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:56:11.292644 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f4ded-407e-4b55-8808-14f38abf62cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:56:11.292910 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f4ded-407e-4b55-8808-14f38abf62cc admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:56:11.293565 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f4ded-407e-4b55-8808-14f38abf62cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:56:11.294568 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-665f4ded-407e-4b55-8808-14f38abf62cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 04:56:11.294568 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jul 26 04:56:11.294568 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: Jul 26 04:56:11.295801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f4ded-407e-4b55-8808-14f38abf62cc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:56:11.347745 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 740/1498] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 04:56:11 2022] POST /identity/v3/users => generated 289 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:56:11.352380 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5a37c-5fbf-430c-981b-b7bac19a29df admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:56:11.353911 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-1ed5a37c-5fbf-430c-981b-b7bac19a29df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:56:11.418117 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5a37c-5fbf-430c-981b-b7bac19a29df admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:56:11.516768 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:56:11.529185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d82e2-77da-457c-b894-11fb68470479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:56:11.529833 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d82e2-77da-457c-b894-11fb68470479 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:56:11.530099 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d82e2-77da-457c-b894-11fb68470479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:56:11.530370 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d82e2-77da-457c-b894-11fb68470479 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:56:11.531089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d82e2-77da-457c-b894-11fb68470479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:56:11.532182 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: WARNING py.warnings [None req-499d82e2-77da-457c-b894-11fb68470479 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 04:56:11.532182 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: warnings.warn(msg) Jul 26 04:56:11.532182 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: Jul 26 04:56:11.533122 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d82e2-77da-457c-b894-11fb68470479 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:56:11.546506 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 759/1499] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 04:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 04:56:11.552069 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f4ded-407e-4b55-8808-14f38abf62cc admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:56:11.553630 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-665f4ded-407e-4b55-8808-14f38abf62cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:56:11.565529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f4ded-407e-4b55-8808-14f38abf62cc admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:56:11.592997 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:56:11.679949 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a9107-781e-48ee-bed0-fc4fb604ace9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:56:11.680591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a9107-781e-48ee-bed0-fc4fb604ace9 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:56:11.680936 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a9107-781e-48ee-bed0-fc4fb604ace9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:56:11.681216 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a9107-781e-48ee-bed0-fc4fb604ace9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:56:11.681935 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a9107-781e-48ee-bed0-fc4fb604ace9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:56:11.682962 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-d99a9107-781e-48ee-bed0-fc4fb604ace9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 04:56:11.682962 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jul 26 04:56:11.682962 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: Jul 26 04:56:11.683890 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a9107-781e-48ee-bed0-fc4fb604ace9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:56:11.697809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 741/1500] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 04:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 04:56:11.702624 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d82e2-77da-457c-b894-11fb68470479 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:56:11.704902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-499d82e2-77da-457c-b894-11fb68470479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:56:11.742964 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d82e2-77da-457c-b894-11fb68470479 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:56:11.846745 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7716815-9633-4ae7-a789-040f3bdb892f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:56:11.847347 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7716815-9633-4ae7-a789-040f3bdb892f admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:56:11.847664 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7716815-9633-4ae7-a789-040f3bdb892f admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:56:11.847941 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7716815-9633-4ae7-a789-040f3bdb892f admin admin] PATH_INFO: `/v3/projects/d3bcec8ca1754bd3a6c63fe7fb666c0c/users/5a08d33340ba4290becaee77174f9d91/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:56:11.874135 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7716815-9633-4ae7-a789-040f3bdb892f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3bcec8ca1754bd3a6c63fe7fb666c0c, user_id=5a08d33340ba4290becaee77174f9d91, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:56:11.875219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: WARNING py.warnings [None req-c7716815-9633-4ae7-a789-040f3bdb892f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 04:56:11.875219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: warnings.warn(msg) Jul 26 04:56:11.875219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: Jul 26 04:56:11.877263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7716815-9633-4ae7-a789-040f3bdb892f admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:56:11.906639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 760/1501] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 04:56:11 2022] PUT /identity/v3/projects/d3bcec8ca1754bd3a6c63fe7fb666c0c/users/5a08d33340ba4290becaee77174f9d91/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:56:11.922072 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a02b12-e245-4ba8-b82e-0f9dc788ebd9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:56:11.922424 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a02b12-e245-4ba8-b82e-0f9dc788ebd9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:56:11.922818 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a02b12-e245-4ba8-b82e-0f9dc788ebd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:56:11.992280 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-04a02b12-e245-4ba8-b82e-0f9dc788ebd9 None None] MFA Rules not processed for user `5a08d33340ba4290becaee77174f9d91`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:56:11.993870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-04a02b12-e245-4ba8-b82e-0f9dc788ebd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:56:12.094240 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 742/1502] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:56:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:56:12.109672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cb76b-af9d-4826-a9be-a335a2ee3059 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:56:12.109957 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cb76b-af9d-4826-a9be-a335a2ee3059 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:56:12.110228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cb76b-af9d-4826-a9be-a335a2ee3059 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:56:12.111648 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 761/1503] 10.222.0.17 () {54 vars in 964 bytes} [Tue Jul 26 04:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:56:12.117226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb6ba5-0c3e-4179-b1bc-02fb9cf71346 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:56:12.117510 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb6ba5-0c3e-4179-b1bc-02fb9cf71346 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:56:12.118420 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb6ba5-0c3e-4179-b1bc-02fb9cf71346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:56:12.151780 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:56:12.204470 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e8cb6ba5-0c3e-4179-b1bc-02fb9cf71346 None None] MFA Rules not processed for user `79ef3d83ac454fac9f890374cd22507a`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:56:12.205727 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e8cb6ba5-0c3e-4179-b1bc-02fb9cf71346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:56:12.303666 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 743/1504] 10.222.0.17 () {58 vars in 1061 bytes} [Tue Jul 26 04:56:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:56:12.313508 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449cb76b-af9d-4826-a9be-a335a2ee3059 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:56:12.315073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-449cb76b-af9d-4826-a9be-a335a2ee3059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:56:12.370071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:56:12.437622 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449cb76b-af9d-4826-a9be-a335a2ee3059 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:56:12.530855 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692fdca-0237-4989-b65a-00bee394f73d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:56:12.531475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692fdca-0237-4989-b65a-00bee394f73d service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:56:12.531803 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692fdca-0237-4989-b65a-00bee394f73d service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:56:12.532084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692fdca-0237-4989-b65a-00bee394f73d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:56:12.533773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c692fdca-0237-4989-b65a-00bee394f73d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:56:12.641681 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c692fdca-0237-4989-b65a-00bee394f73d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:56:12.643118 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c692fdca-0237-4989-b65a-00bee394f73d service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:56:12.647392 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:56:12.666668 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 762/1505] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:56:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:56:12.719799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c6e0b-7535-4cb7-8230-0ea5af9ff1e2 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:56:12.720371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c6e0b-7535-4cb7-8230-0ea5af9ff1e2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:56:12.720763 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c6e0b-7535-4cb7-8230-0ea5af9ff1e2 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:56:12.723215 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 744/1506] 10.222.0.17 () {52 vars in 830 bytes} [Tue Jul 26 04:56:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:56:12.728376 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692fdca-0237-4989-b65a-00bee394f73d service cinder] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:56:12.729695 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c692fdca-0237-4989-b65a-00bee394f73d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:56:12.855099 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692fdca-0237-4989-b65a-00bee394f73d service cinder] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:56:12.978643 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6874686f-c29c-47e3-8fc3-1d88c184edd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:56:12.979387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874686f-c29c-47e3-8fc3-1d88c184edd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:56:12.979755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874686f-c29c-47e3-8fc3-1d88c184edd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:56:12.980036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874686f-c29c-47e3-8fc3-1d88c184edd5 admin admin] PATH_INFO: `/v3/users/5a08d33340ba4290becaee77174f9d91` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:56:12.996394 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6874686f-c29c-47e3-8fc3-1d88c184edd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a08d33340ba4290becaee77174f9d91)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:56:12.997882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: WARNING py.warnings [None req-6874686f-c29c-47e3-8fc3-1d88c184edd5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 04:56:12.997882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: warnings.warn(msg) Jul 26 04:56:12.997882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: Jul 26 04:56:12.999593 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6874686f-c29c-47e3-8fc3-1d88c184edd5 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:56:13.012819 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:56:13.080778 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6874686f-c29c-47e3-8fc3-1d88c184edd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a08d33340ba4290becaee77174f9d91'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:56:13.106337 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6874686f-c29c-47e3-8fc3-1d88c184edd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a08d33340ba4290becaee77174f9d91'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:56:13.118377 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6874686f-c29c-47e3-8fc3-1d88c184edd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a08d33340ba4290becaee77174f9d91'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:56:13.119037 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6874686f-c29c-47e3-8fc3-1d88c184edd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a08d33340ba4290becaee77174f9d91'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:56:13.119713 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6874686f-c29c-47e3-8fc3-1d88c184edd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a08d33340ba4290becaee77174f9d91'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:56:13.148394 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 763/1507] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 04:56:12 2022] DELETE /identity/v3/users/5a08d33340ba4290becaee77174f9d91 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:56:13.153097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1c14e-1ff1-4d57-af35-cdcf895fd440 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:56:13.153565 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1c14e-1ff1-4d57-af35-cdcf895fd440 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:56:13.153849 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1c14e-1ff1-4d57-af35-cdcf895fd440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:56:13.223345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-02c1c14e-1ff1-4d57-af35-cdcf895fd440 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:56:13.224702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-02c1c14e-1ff1-4d57-af35-cdcf895fd440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:56:13.308267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:56:13.324271 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 745/1508] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:56:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:56:13.338791 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6874686f-c29c-47e3-8fc3-1d88c184edd5 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:56:13.340370 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-6874686f-c29c-47e3-8fc3-1d88c184edd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:56:13.457212 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6874686f-c29c-47e3-8fc3-1d88c184edd5 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:56:13.518774 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:56:13.546874 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f41ad-e797-43ee-b151-42802e8b5d2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:56:13.548089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f41ad-e797-43ee-b151-42802e8b5d2b service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:56:13.548668 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f41ad-e797-43ee-b151-42802e8b5d2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:56:13.548981 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f41ad-e797-43ee-b151-42802e8b5d2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:56:13.551170 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-620f41ad-e797-43ee-b151-42802e8b5d2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:56:13.686838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f41ad-e797-43ee-b151-42802e8b5d2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:56:13.688376 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f41ad-e797-43ee-b151-42802e8b5d2b service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:56:13.707762 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 764/1509] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:56:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:56:14.118657 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1c14e-1ff1-4d57-af35-cdcf895fd440 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:56:14.119934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-02c1c14e-1ff1-4d57-af35-cdcf895fd440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:56:14.131661 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1c14e-1ff1-4d57-af35-cdcf895fd440 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:56:14.252331 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16a790-9e80-4285-81b7-abd164496f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:56:14.255092 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16a790-9e80-4285-81b7-abd164496f34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:56:14.255092 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16a790-9e80-4285-81b7-abd164496f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:56:14.255092 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16a790-9e80-4285-81b7-abd164496f34 admin admin] PATH_INFO: `/v3/projects/d3bcec8ca1754bd3a6c63fe7fb666c0c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:56:14.267203 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16a790-9e80-4285-81b7-abd164496f34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3bcec8ca1754bd3a6c63fe7fb666c0c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:56:14.268290 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-dd16a790-9e80-4285-81b7-abd164496f34 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 04:56:14.268290 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jul 26 04:56:14.268290 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: Jul 26 04:56:14.269503 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16a790-9e80-4285-81b7-abd164496f34 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:56:14.330675 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dd16a790-9e80-4285-81b7-abd164496f34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3bcec8ca1754bd3a6c63fe7fb666c0c', 'request_id': 'req-dd16a790-9e80-4285-81b7-abd164496f34'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:56:14.344964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dd16a790-9e80-4285-81b7-abd164496f34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3bcec8ca1754bd3a6c63fe7fb666c0c', 'request_id': 'req-dd16a790-9e80-4285-81b7-abd164496f34'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:56:14.356483 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dd16a790-9e80-4285-81b7-abd164496f34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3bcec8ca1754bd3a6c63fe7fb666c0c', 'request_id': 'req-dd16a790-9e80-4285-81b7-abd164496f34'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:56:14.380185 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dd16a790-9e80-4285-81b7-abd164496f34 admin admin] The token cache is being invalidate because project d3bcec8ca1754bd3a6c63fe7fb666c0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 04:56:14.380533 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dd16a790-9e80-4285-81b7-abd164496f34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:56:14.381143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dd16a790-9e80-4285-81b7-abd164496f34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:56:14.382523 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 746/1510] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 04:56:14 2022] DELETE /identity/v3/projects/d3bcec8ca1754bd3a6c63fe7fb666c0c => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:56:23.705416 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=62 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:56:24.378419 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:57:52.103721 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae772d5e-8d46-40ad-990e-d6e94c2d96eb None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:57:52.104414 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae772d5e-8d46-40ad-990e-d6e94c2d96eb None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:57:52.104414 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae772d5e-8d46-40ad-990e-d6e94c2d96eb None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:57:52.105715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 765/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 04:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:57:52.111956 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e754336-c2db-4aa9-af3f-b52952dd7ee6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:57:52.112549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e754336-c2db-4aa9-af3f-b52952dd7ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:57:52.112549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e754336-c2db-4aa9-af3f-b52952dd7ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:57:52.212926 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7e754336-c2db-4aa9-af3f-b52952dd7ee6 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:57:52.214185 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7e754336-c2db-4aa9-af3f-b52952dd7ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:57:52.365957 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 747/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 04:57:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:57:52.520961 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae772d5e-8d46-40ad-990e-d6e94c2d96eb None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:57:52.523479 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-ae772d5e-8d46-40ad-990e-d6e94c2d96eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:57:52.682276 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae772d5e-8d46-40ad-990e-d6e94c2d96eb None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:57:52.688414 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f65365-ca1d-4316-ad28-accb90ca3589 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:57:52.689059 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f65365-ca1d-4316-ad28-accb90ca3589 service nova] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:57:52.689344 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f65365-ca1d-4316-ad28-accb90ca3589 service nova] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:57:52.689616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f65365-ca1d-4316-ad28-accb90ca3589 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:57:52.698966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f65365-ca1d-4316-ad28-accb90ca3589 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:57:52.700799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f65365-ca1d-4316-ad28-accb90ca3589 service nova] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:57:52.711960 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 766/1513] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:57:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:57:53.563842 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f04f5-e203-4bcf-9309-63dbb72dbb87 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:57:53.564498 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f04f5-e203-4bcf-9309-63dbb72dbb87 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:57:53.564751 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f04f5-e203-4bcf-9309-63dbb72dbb87 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:57:53.566854 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 748/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 04:57:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:57:53.573996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e026fa-7bf1-4c09-88ec-c3bffed9811b None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:57:53.574625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e026fa-7bf1-4c09-88ec-c3bffed9811b None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:57:53.574625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e026fa-7bf1-4c09-88ec-c3bffed9811b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:57:53.598791 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-d8e026fa-7bf1-4c09-88ec-c3bffed9811b None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:57:53.600861 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-d8e026fa-7bf1-4c09-88ec-c3bffed9811b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:57:53.611470 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 767/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 04:57:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:57:53.767504 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f04f5-e203-4bcf-9309-63dbb72dbb87 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:57:53.784459 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f04f5-e203-4bcf-9309-63dbb72dbb87 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:57:53.790735 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ea06e-8e5a-4d28-8aee-eb406ebfb340 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:57:53.791384 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ea06e-8e5a-4d28-8aee-eb406ebfb340 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:57:53.791739 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ea06e-8e5a-4d28-8aee-eb406ebfb340 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:57:53.792013 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ea06e-8e5a-4d28-8aee-eb406ebfb340 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:57:53.802251 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156ea06e-8e5a-4d28-8aee-eb406ebfb340 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:57:53.804013 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156ea06e-8e5a-4d28-8aee-eb406ebfb340 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:57:53.815730 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 749/1516] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:57:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:57:53.947299 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b951bc-7fe9-4274-8c0c-6e3e56e78d46 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:57:53.947661 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b951bc-7fe9-4274-8c0c-6e3e56e78d46 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:57:53.947905 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b951bc-7fe9-4274-8c0c-6e3e56e78d46 None None] PATH_INFO: `/v3` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:57:53.949376 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 768/1517] 10.222.0.52 () {54 vars in 925 bytes} [Tue Jul 26 04:57:53 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:57:53.957642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c6ed7-e286-4955-b200-b1ca01e1ba18 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:57:53.958003 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c6ed7-e286-4955-b200-b1ca01e1ba18 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:57:53.958328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c6ed7-e286-4955-b200-b1ca01e1ba18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:57:54.041473 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-582c6ed7-e286-4955-b200-b1ca01e1ba18 None None] MFA Rules not processed for user `5049dba7ffde47b8af3e6fd55b5703e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:57:54.042824 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-582c6ed7-e286-4955-b200-b1ca01e1ba18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:57:54.062360 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:57:54.125332 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 750/1518] 10.222.0.52 () {58 vars in 1015 bytes} [Tue Jul 26 04:57:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:58:03.069463 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60390652-4ac9-420a-a366-fe51195197d8 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:58:03.069463 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60390652-4ac9-420a-a366-fe51195197d8 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:58:03.070104 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60390652-4ac9-420a-a366-fe51195197d8 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:58:03.071523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 769/1519] 10.222.0.17 () {54 vars in 961 bytes} [Tue Jul 26 04:58:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:58:03.076714 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d3be3-117c-4477-890c-d4b926b0fed2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:58:03.077198 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d3be3-117c-4477-890c-d4b926b0fed2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:58:03.077309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d3be3-117c-4477-890c-d4b926b0fed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:58:03.098885 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ad0d3be3-117c-4477-890c-d4b926b0fed2 None None] MFA Rules not processed for user `5049dba7ffde47b8af3e6fd55b5703e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:58:03.100135 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ad0d3be3-117c-4477-890c-d4b926b0fed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:58:03.112506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 751/1520] 10.222.0.17 () {58 vars in 1058 bytes} [Tue Jul 26 04:58:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:58:03.123411 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60390652-4ac9-420a-a366-fe51195197d8 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:58:03.138851 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60390652-4ac9-420a-a366-fe51195197d8 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:58:03.144774 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107560a-e0b7-4871-9038-f455cfa43a82 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5049dba7ffde47b8af3e6fd55b5703e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:58:03.145406 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107560a-e0b7-4871-9038-f455cfa43a82 service placement] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:58:03.145671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107560a-e0b7-4871-9038-f455cfa43a82 service placement] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:58:03.145946 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107560a-e0b7-4871-9038-f455cfa43a82 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:58:03.155065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4107560a-e0b7-4871-9038-f455cfa43a82 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:58:03.156726 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4107560a-e0b7-4871-9038-f455cfa43a82 service placement] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:58:03.166731 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 770/1521] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:58:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:58:05.112677 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4facb89c-8932-4124-b20e-acb6226dd83d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:58:05.113265 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4facb89c-8932-4124-b20e-acb6226dd83d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:58:05.113265 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4facb89c-8932-4124-b20e-acb6226dd83d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:58:05.114620 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 752/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 04:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:58:05.119842 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf370f-1281-4374-a842-0138e74f3aac None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:58:05.120332 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf370f-1281-4374-a842-0138e74f3aac None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:58:05.120332 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf370f-1281-4374-a842-0138e74f3aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:58:05.140942 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-76bf370f-1281-4374-a842-0138e74f3aac None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:58:05.142153 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-76bf370f-1281-4374-a842-0138e74f3aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:58:05.150374 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 771/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 04:58:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:58:05.324674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4facb89c-8932-4124-b20e-acb6226dd83d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:58:05.340242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4facb89c-8932-4124-b20e-acb6226dd83d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:58:05.346412 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25e7b7-062d-47bd-93d1-bb95f1bb7169 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:58:05.347058 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25e7b7-062d-47bd-93d1-bb95f1bb7169 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:58:05.347341 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25e7b7-062d-47bd-93d1-bb95f1bb7169 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:58:05.347652 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25e7b7-062d-47bd-93d1-bb95f1bb7169 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:58:05.357819 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab25e7b7-062d-47bd-93d1-bb95f1bb7169 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:58:05.359912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab25e7b7-062d-47bd-93d1-bb95f1bb7169 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:58:05.371376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 753/1524] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:58:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:58:06.192146 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd8d60-8956-46d9-a10d-8f82cd657510 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:58:06.193043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd8d60-8956-46d9-a10d-8f82cd657510 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:58:06.193043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd8d60-8956-46d9-a10d-8f82cd657510 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:58:06.195132 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 772/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 04:58:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:58:06.203290 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff0cab-0d57-490a-b288-9e2f959dc4ea None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:58:06.203916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff0cab-0d57-490a-b288-9e2f959dc4ea None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:58:06.203916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff0cab-0d57-490a-b288-9e2f959dc4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:58:06.225128 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-75ff0cab-0d57-490a-b288-9e2f959dc4ea None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:58:06.226344 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-75ff0cab-0d57-490a-b288-9e2f959dc4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:58:06.235526 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 754/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 04:58:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:58:06.393232 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd8d60-8956-46d9-a10d-8f82cd657510 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:58:06.409792 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd8d60-8956-46d9-a10d-8f82cd657510 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:58:06.417290 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d2331-3b05-4c24-93ba-5ff91083c365 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:58:06.417910 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d2331-3b05-4c24-93ba-5ff91083c365 service nova] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:58:06.418188 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d2331-3b05-4c24-93ba-5ff91083c365 service nova] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:58:06.418461 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d2331-3b05-4c24-93ba-5ff91083c365 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:58:06.427705 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d2331-3b05-4c24-93ba-5ff91083c365 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:58:06.429288 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d2331-3b05-4c24-93ba-5ff91083c365 service nova] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:58:06.439925 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 773/1527] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:58:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 04:58:10.362141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde52ca8-c706-4da6-9a61-4d1985c1ce69 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:58:10.362759 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde52ca8-c706-4da6-9a61-4d1985c1ce69 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:58:10.362759 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde52ca8-c706-4da6-9a61-4d1985c1ce69 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:58:10.364335 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 755/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 04:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:58:10.370205 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3292dc-3be0-4f5e-8ca3-b6e0f354a7a7 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:58:10.370772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3292dc-3be0-4f5e-8ca3-b6e0f354a7a7 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:58:10.370772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3292dc-3be0-4f5e-8ca3-b6e0f354a7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:58:10.393149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-ec3292dc-3be0-4f5e-8ca3-b6e0f354a7a7 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:58:10.394383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-ec3292dc-3be0-4f5e-8ca3-b6e0f354a7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:58:10.402531 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 774/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 04:58:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 04:58:10.484966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde52ca8-c706-4da6-9a61-4d1985c1ce69 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:58:10.487578 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bde52ca8-c706-4da6-9a61-4d1985c1ce69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:58:10.580701 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde52ca8-c706-4da6-9a61-4d1985c1ce69 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:58:10.585088 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9e303-dc29-4127-885f-9bc514b40b1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:58:10.585717 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9e303-dc29-4127-885f-9bc514b40b1a service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:58:10.586000 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9e303-dc29-4127-885f-9bc514b40b1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:58:10.586275 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9e303-dc29-4127-885f-9bc514b40b1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:58:10.596229 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9e303-dc29-4127-885f-9bc514b40b1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:58:10.597696 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9e303-dc29-4127-885f-9bc514b40b1a service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:58:10.607745 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 756/1530] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:58:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:58:12.654032 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4103cb-0bb2-49cf-a588-2c83405c25ed None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:58:12.655832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4103cb-0bb2-49cf-a588-2c83405c25ed None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:58:12.655832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4103cb-0bb2-49cf-a588-2c83405c25ed None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:58:12.657331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 775/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 04:58:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:58:12.664914 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1afb47-f7c4-44d1-85da-737104a4b019 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:58:12.665756 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1afb47-f7c4-44d1-85da-737104a4b019 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:58:12.665756 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1afb47-f7c4-44d1-85da-737104a4b019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:58:12.688726 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3f1afb47-f7c4-44d1-85da-737104a4b019 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:58:12.690058 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3f1afb47-f7c4-44d1-85da-737104a4b019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:58:12.699510 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 757/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 04:58:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 04:58:12.750942 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4103cb-0bb2-49cf-a588-2c83405c25ed None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:58:12.752994 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-bc4103cb-0bb2-49cf-a588-2c83405c25ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:58:12.768535 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4103cb-0bb2-49cf-a588-2c83405c25ed None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:58:12.772729 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a586224-06a1-41c0-bc49-7a662ddbd7ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:58:12.773527 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a586224-06a1-41c0-bc49-7a662ddbd7ea service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:58:12.773899 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a586224-06a1-41c0-bc49-7a662ddbd7ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:58:12.774286 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a586224-06a1-41c0-bc49-7a662ddbd7ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:58:12.785815 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a586224-06a1-41c0-bc49-7a662ddbd7ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:58:12.787321 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a586224-06a1-41c0-bc49-7a662ddbd7ea service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:58:12.798602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 776/1533] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:58:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:58:16.533708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9b1ba-c5a3-4322-9e46-c877f5c28736 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:58:16.534308 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9b1ba-c5a3-4322-9e46-c877f5c28736 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:58:16.534540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9b1ba-c5a3-4322-9e46-c877f5c28736 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:58:16.536662 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 758/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 04:58:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:58:16.542848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad331d-cafc-4121-9669-a7e9ba2624b7 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:58:16.543385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad331d-cafc-4121-9669-a7e9ba2624b7 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:58:16.543385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad331d-cafc-4121-9669-a7e9ba2624b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:58:16.551718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=81 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:58:16.575182 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-2aad331d-cafc-4121-9669-a7e9ba2624b7 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:58:16.577257 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-2aad331d-cafc-4121-9669-a7e9ba2624b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:58:16.594465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 777/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 04:58:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:58:16.912668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c9b1ba-c5a3-4322-9e46-c877f5c28736 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:58:16.914501 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-52c9b1ba-c5a3-4322-9e46-c877f5c28736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:58:16.997175 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:58:17.015086 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c9b1ba-c5a3-4322-9e46-c877f5c28736 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:58:17.019025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfbe3f7-d5e4-48cf-917c-7a848cc40851 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:58:17.019711 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfbe3f7-d5e4-48cf-917c-7a848cc40851 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:58:17.019997 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfbe3f7-d5e4-48cf-917c-7a848cc40851 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:58:17.020273 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfbe3f7-d5e4-48cf-917c-7a848cc40851 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:58:17.031034 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfbe3f7-d5e4-48cf-917c-7a848cc40851 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:58:17.032586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfbe3f7-d5e4-48cf-917c-7a848cc40851 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:58:17.043545 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 759/1536] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:58:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:58:19.172188 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9849a70-64d9-4806-9e1e-dc573ee28597 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:58:19.172762 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9849a70-64d9-4806-9e1e-dc573ee28597 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:58:19.172762 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9849a70-64d9-4806-9e1e-dc573ee28597 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:58:19.174117 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 778/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 04:58:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:58:19.179815 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ab7b3-7958-47dc-8646-49cb015b94a6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:58:19.180308 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ab7b3-7958-47dc-8646-49cb015b94a6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:58:19.180402 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ab7b3-7958-47dc-8646-49cb015b94a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:58:19.202603 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c80ab7b3-7958-47dc-8646-49cb015b94a6 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:58:19.203977 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c80ab7b3-7958-47dc-8646-49cb015b94a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:58:19.212812 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 760/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 04:58:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:58:19.528026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9849a70-64d9-4806-9e1e-dc573ee28597 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:58:19.529831 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-f9849a70-64d9-4806-9e1e-dc573ee28597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:58:19.543688 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9849a70-64d9-4806-9e1e-dc573ee28597 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:58:19.547297 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398bf0e-4974-4d07-baf0-e3c4b99efc69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:58:19.547969 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398bf0e-4974-4d07-baf0-e3c4b99efc69 service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:58:19.548250 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398bf0e-4974-4d07-baf0-e3c4b99efc69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:58:19.548535 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398bf0e-4974-4d07-baf0-e3c4b99efc69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:58:19.558523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f398bf0e-4974-4d07-baf0-e3c4b99efc69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:58:19.560057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f398bf0e-4974-4d07-baf0-e3c4b99efc69 service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:58:19.571260 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 779/1539] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:58:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:58:29.197331 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=11 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:58:29.566713 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=8 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:59:49.646529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3eedf-68c5-4510-bc2d-b1534f8fcfc5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:49.647168 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3eedf-68c5-4510-bc2d-b1534f8fcfc5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:49.647457 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3eedf-68c5-4510-bc2d-b1534f8fcfc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:49.698373 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e7b3eedf-68c5-4510-bc2d-b1534f8fcfc5 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:49.700421 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e7b3eedf-68c5-4510-bc2d-b1534f8fcfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:49.728801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 761/1540] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:49.739823 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae30839-d23d-47a6-a662-47bbea80e169 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:49.740519 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae30839-d23d-47a6-a662-47bbea80e169 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:49.740845 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae30839-d23d-47a6-a662-47bbea80e169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:49.787145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33905e8e-b3a9-45c3-96b2-a3e93a1ccc50 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:49.787695 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33905e8e-b3a9-45c3-96b2-a3e93a1ccc50 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:49.788198 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33905e8e-b3a9-45c3-96b2-a3e93a1ccc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:49.789432 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-4ae30839-d23d-47a6-a662-47bbea80e169 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:49.791664 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-4ae30839-d23d-47a6-a662-47bbea80e169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:49.821047 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 780/1541] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:49.827081 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae30839-d23d-47a6-a662-47bbea80e169 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:49.845772 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-33905e8e-b3a9-45c3-96b2-a3e93a1ccc50 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:49.852172 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-33905e8e-b3a9-45c3-96b2-a3e93a1ccc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:49.856711 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae30839-d23d-47a6-a662-47bbea80e169 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:49.874158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 762/1542] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 04:59:49.877675 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dacf95-5e47-4a46-84f4-7638d82b3a79 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:49.878159 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dacf95-5e47-4a46-84f4-7638d82b3a79 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:49.878646 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dacf95-5e47-4a46-84f4-7638d82b3a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:49.880897 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d22ef-93aa-4913-b354-c9ee4ec8e582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:49.881956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d22ef-93aa-4913-b354-c9ee4ec8e582 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:49.882440 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d22ef-93aa-4913-b354-c9ee4ec8e582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:49.882944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d22ef-93aa-4913-b354-c9ee4ec8e582 admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:49.884342 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d22ef-93aa-4913-b354-c9ee4ec8e582 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:59:49.885056 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d22ef-93aa-4913-b354-c9ee4ec8e582 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:49.886996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: WARNING py.warnings [None req-842d22ef-93aa-4913-b354-c9ee4ec8e582 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 04:59:49.886996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: warnings.warn(msg) Jul 26 04:59:49.886996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: Jul 26 04:59:49.888735 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d22ef-93aa-4913-b354-c9ee4ec8e582 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:49.928555 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 781/1543] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 04:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:59:49.930872 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99469a87-dcf0-45ad-8a00-a476b043345d None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:49.931211 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99469a87-dcf0-45ad-8a00-a476b043345d None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:49.935180 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99469a87-dcf0-45ad-8a00-a476b043345d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:49.937448 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b2dacf95-5e47-4a46-84f4-7638d82b3a79 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:49.939085 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b2dacf95-5e47-4a46-84f4-7638d82b3a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:49.972709 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 763/1544] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:49.975316 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8ffd7-fb25-4576-9c1e-f0ff80d96a83 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:49.975863 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8ffd7-fb25-4576-9c1e-f0ff80d96a83 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:49.976366 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8ffd7-fb25-4576-9c1e-f0ff80d96a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:49.996076 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-99469a87-dcf0-45ad-8a00-a476b043345d None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:49.997949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-99469a87-dcf0-45ad-8a00-a476b043345d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:50.017095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 782/1545] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:50.018384 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99469a87-dcf0-45ad-8a00-a476b043345d None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:50.029728 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99469a87-dcf0-45ad-8a00-a476b043345d None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:50.031010 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3ca8ffd7-fb25-4576-9c1e-f0ff80d96a83 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:50.036889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3ca8ffd7-fb25-4576-9c1e-f0ff80d96a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:50.048479 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c285a-9f29-4d6e-a906-577f9c8d0a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:50.050336 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c285a-9f29-4d6e-a906-577f9c8d0a58 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:50.052155 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c285a-9f29-4d6e-a906-577f9c8d0a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:50.052919 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c285a-9f29-4d6e-a906-577f9c8d0a58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:50.055186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 764/1546] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:50.056876 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c285a-9f29-4d6e-a906-577f9c8d0a58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:50.060693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504ac1c-40f5-4891-a53d-0bfa69609617 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:50.064052 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504ac1c-40f5-4891-a53d-0bfa69609617 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:50.064672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504ac1c-40f5-4891-a53d-0bfa69609617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:50.065331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c285a-9f29-4d6e-a906-577f9c8d0a58 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:50.092525 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8504ac1c-40f5-4891-a53d-0bfa69609617 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:50.094191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8504ac1c-40f5-4891-a53d-0bfa69609617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:50.104000 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 765/1547] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:50.105510 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a64d5-ceec-4c21-b2fd-3b30150f361d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:50.105918 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a64d5-ceec-4c21-b2fd-3b30150f361d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:50.106296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a64d5-ceec-4c21-b2fd-3b30150f361d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:50.131907 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-628a64d5-ceec-4c21-b2fd-3b30150f361d None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:50.133782 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-628a64d5-ceec-4c21-b2fd-3b30150f361d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:50.149633 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 766/1548] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:50.150683 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a64d5-ceec-4c21-b2fd-3b30150f361d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:50.161193 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 783/1549] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 04:59:50 2022] POST /identity/v3/projects => generated 403 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:59:50.161193 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01ea4b-d7cb-4934-a625-80ef06c9535a None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:50.161193 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01ea4b-d7cb-4934-a625-80ef06c9535a None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:50.161193 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01ea4b-d7cb-4934-a625-80ef06c9535a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:50.162878 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a64d5-ceec-4c21-b2fd-3b30150f361d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:50.188947 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-ff01ea4b-d7cb-4934-a625-80ef06c9535a None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:50.190096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-ff01ea4b-d7cb-4934-a625-80ef06c9535a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:50.241953 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a2884-0a66-4ebf-8582-01b73a59d9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:50.242758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a2884-0a66-4ebf-8582-01b73a59d9b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:50.243150 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a2884-0a66-4ebf-8582-01b73a59d9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:50.243504 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a2884-0a66-4ebf-8582-01b73a59d9b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:50.244450 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0a2884-0a66-4ebf-8582-01b73a59d9b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:59:50.244849 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0a2884-0a66-4ebf-8582-01b73a59d9b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:50.246729 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0a2884-0a66-4ebf-8582-01b73a59d9b2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:50.274306 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 784/1550] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:50.275580 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7733bce-84df-4c8b-b30c-08b8e896314b None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:50.275972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7733bce-84df-4c8b-b30c-08b8e896314b None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:50.276371 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7733bce-84df-4c8b-b30c-08b8e896314b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:50.288122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 767/1551] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 04:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:59:50.288122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a2884-0a66-4ebf-8582-01b73a59d9b2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:50.301938 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a2884-0a66-4ebf-8582-01b73a59d9b2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:50.307997 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-b7733bce-84df-4c8b-b30c-08b8e896314b None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:50.308331 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc7f73-f8f1-48b3-8fd8-30bc5a0a1496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:50.308809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc7f73-f8f1-48b3-8fd8-30bc5a0a1496 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:50.309243 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc7f73-f8f1-48b3-8fd8-30bc5a0a1496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:50.309492 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc7f73-f8f1-48b3-8fd8-30bc5a0a1496 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:50.309881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-b7733bce-84df-4c8b-b30c-08b8e896314b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:50.310516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc7f73-f8f1-48b3-8fd8-30bc5a0a1496 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:59:50.310829 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc7f73-f8f1-48b3-8fd8-30bc5a0a1496 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:50.312666 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc7f73-f8f1-48b3-8fd8-30bc5a0a1496 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:50.319363 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 785/1552] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:50.320625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3a932-4ab1-451b-b377-787043e15851 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:50.320989 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3a932-4ab1-451b-b377-787043e15851 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:50.321368 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3a932-4ab1-451b-b377-787043e15851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:50.337683 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 768/1553] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 04:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:59:50.339026 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de293bf-1755-4802-a911-70b68708076a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:50.339026 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de293bf-1755-4802-a911-70b68708076a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:50.339359 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de293bf-1755-4802-a911-70b68708076a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:50.349009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-c2e3a932-4ab1-451b-b377-787043e15851 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:50.351210 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c2e3a932-4ab1-451b-b377-787043e15851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:50.361134 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2de293bf-1755-4802-a911-70b68708076a None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:50.362060 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 786/1554] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:50.362398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2de293bf-1755-4802-a911-70b68708076a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:50.364002 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3a932-4ab1-451b-b377-787043e15851 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:50.378531 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 769/1555] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:50.378531 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de293bf-1755-4802-a911-70b68708076a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:50.381442 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3a932-4ab1-451b-b377-787043e15851 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:50.386544 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de293bf-1755-4802-a911-70b68708076a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:50.387274 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e013d-8207-44d2-95ab-5aa7cc51e1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:50.388308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e013d-8207-44d2-95ab-5aa7cc51e1ed admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:50.388792 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e013d-8207-44d2-95ab-5aa7cc51e1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:50.389268 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e013d-8207-44d2-95ab-5aa7cc51e1ed admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:50.390455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5e013d-8207-44d2-95ab-5aa7cc51e1ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:50.391507 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4a80a-7405-4f35-abfd-14c184c777a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:50.392348 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4a80a-7405-4f35-abfd-14c184c777a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:50.392456 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: WARNING py.warnings [None req-ca5e013d-8207-44d2-95ab-5aa7cc51e1ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 04:59:50.392456 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: warnings.warn(msg) Jul 26 04:59:50.392456 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: Jul 26 04:59:50.392707 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4a80a-7405-4f35-abfd-14c184c777a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:50.392853 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4a80a-7405-4f35-abfd-14c184c777a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:50.393558 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be4a80a-7405-4f35-abfd-14c184c777a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:59:50.393869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be4a80a-7405-4f35-abfd-14c184c777a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:50.394047 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5e013d-8207-44d2-95ab-5aa7cc51e1ed admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:50.395504 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be4a80a-7405-4f35-abfd-14c184c777a7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:50.422010 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 770/1556] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 04:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:59:50.422010 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4a80a-7405-4f35-abfd-14c184c777a7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:50.422504 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 787/1557] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 04:59:50 2022] POST /identity/v3/users => generated 302 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:59:50.422504 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf71ba7-2acc-4687-80a9-e07b2131c380 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:50.422504 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf71ba7-2acc-4687-80a9-e07b2131c380 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:50.422504 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf71ba7-2acc-4687-80a9-e07b2131c380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:50.429547 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4a80a-7405-4f35-abfd-14c184c777a7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:50.434697 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5184af0e-20d3-4dce-8faa-b6efbe27f557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:50.435353 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5184af0e-20d3-4dce-8faa-b6efbe27f557 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:50.435669 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5184af0e-20d3-4dce-8faa-b6efbe27f557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:50.435950 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5184af0e-20d3-4dce-8faa-b6efbe27f557 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:50.436637 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5184af0e-20d3-4dce-8faa-b6efbe27f557 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:50.437693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-5184af0e-20d3-4dce-8faa-b6efbe27f557 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 04:59:50.437693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jul 26 04:59:50.437693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: Jul 26 04:59:50.438918 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5184af0e-20d3-4dce-8faa-b6efbe27f557 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:50.441180 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-bdf71ba7-2acc-4687-80a9-e07b2131c380 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:50.442349 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-bdf71ba7-2acc-4687-80a9-e07b2131c380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:50.450704 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 788/1558] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:50.452215 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80368452-f3c8-4332-a4b9-e0da4abd1da7 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:50.453723 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80368452-f3c8-4332-a4b9-e0da4abd1da7 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:50.453723 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80368452-f3c8-4332-a4b9-e0da4abd1da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:50.478740 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-80368452-f3c8-4332-a4b9-e0da4abd1da7 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:50.480165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-80368452-f3c8-4332-a4b9-e0da4abd1da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:50.482330 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 771/1559] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 04:59:50 2022] POST /identity/v3/projects => generated 375 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:59:50.483171 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5184af0e-20d3-4dce-8faa-b6efbe27f557 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:50.561532 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5184af0e-20d3-4dce-8faa-b6efbe27f557 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:50.567679 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960b7f9-6a8d-4f9d-b018-85e64b72a532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:50.568964 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 789/1560] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:50.569650 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3696e-97a1-46b0-b77f-eff02213c084 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:50.570210 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3696e-97a1-46b0-b77f-eff02213c084 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:50.570758 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3696e-97a1-46b0-b77f-eff02213c084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:50.571242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960b7f9-6a8d-4f9d-b018-85e64b72a532 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:50.571638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960b7f9-6a8d-4f9d-b018-85e64b72a532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:50.571848 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960b7f9-6a8d-4f9d-b018-85e64b72a532 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:50.572529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960b7f9-6a8d-4f9d-b018-85e64b72a532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:50.574523 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960b7f9-6a8d-4f9d-b018-85e64b72a532 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:50.580297 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:59:50.592508 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:59:50.597131 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-cff3696e-97a1-46b0-b77f-eff02213c084 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:50.599278 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-cff3696e-97a1-46b0-b77f-eff02213c084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:50.613619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 790/1561] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:50.613619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3696e-97a1-46b0-b77f-eff02213c084 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:50.617309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 772/1562] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 04:59:50 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:59:50.618408 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960b7f9-6a8d-4f9d-b018-85e64b72a532 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:50.627309 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3696e-97a1-46b0-b77f-eff02213c084 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:50.698808 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1006ba69-9876-4183-a43b-89455452c29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:50.699347 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006ba69-9876-4183-a43b-89455452c29f admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:50.699722 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006ba69-9876-4183-a43b-89455452c29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:50.700009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006ba69-9876-4183-a43b-89455452c29f admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:50.700384 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960b7f9-6a8d-4f9d-b018-85e64b72a532 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:50.701182 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1006ba69-9876-4183-a43b-89455452c29f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:59:50.701182 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1006ba69-9876-4183-a43b-89455452c29f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:50.702757 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1006ba69-9876-4183-a43b-89455452c29f admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:50.706632 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0edd952-af3c-4218-b77c-bf65711d8aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:50.707612 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edd952-af3c-4218-b77c-bf65711d8aaa admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:50.708149 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edd952-af3c-4218-b77c-bf65711d8aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:50.708540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edd952-af3c-4218-b77c-bf65711d8aaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:50.709441 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0edd952-af3c-4218-b77c-bf65711d8aaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:50.711460 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0edd952-af3c-4218-b77c-bf65711d8aaa admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:50.718773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 791/1563] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 04:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:59:50.719709 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1006ba69-9876-4183-a43b-89455452c29f admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:50.734532 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 773/1564] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 04:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 04:59:50.734770 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1006ba69-9876-4183-a43b-89455452c29f admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:50.734927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a226417-8390-41f4-9b48-98abe98fbc52 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:50.734927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a226417-8390-41f4-9b48-98abe98fbc52 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:50.734927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a226417-8390-41f4-9b48-98abe98fbc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:50.738588 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db05e9-92f1-4d4a-be4b-72f4beeddd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:50.739669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db05e9-92f1-4d4a-be4b-72f4beeddd57 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:50.740203 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db05e9-92f1-4d4a-be4b-72f4beeddd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:50.740981 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db05e9-92f1-4d4a-be4b-72f4beeddd57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:50.741483 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6db05e9-92f1-4d4a-be4b-72f4beeddd57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:50.744456 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6db05e9-92f1-4d4a-be4b-72f4beeddd57 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:50.764367 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7a226417-8390-41f4-9b48-98abe98fbc52 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:50.766193 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7a226417-8390-41f4-9b48-98abe98fbc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:50.780755 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 774/1565] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:50.784134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 792/1566] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 04:59:50 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:59:50.785184 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db05e9-92f1-4d4a-be4b-72f4beeddd57 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:50.788880 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a226417-8390-41f4-9b48-98abe98fbc52 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:50.803066 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a226417-8390-41f4-9b48-98abe98fbc52 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:50.863570 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db05e9-92f1-4d4a-be4b-72f4beeddd57 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:50.869423 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99535e-1f64-4372-9627-0e70d94562f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:50.870110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99535e-1f64-4372-9627-0e70d94562f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:50.870409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99535e-1f64-4372-9627-0e70d94562f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:50.870771 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99535e-1f64-4372-9627-0e70d94562f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:50.871605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99535e-1f64-4372-9627-0e70d94562f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:59:50.871927 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99535e-1f64-4372-9627-0e70d94562f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:50.873592 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99535e-1f64-4372-9627-0e70d94562f9 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:50.894443 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 793/1567] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 04:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:59:50.894443 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99535e-1f64-4372-9627-0e70d94562f9 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:50.902178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99535e-1f64-4372-9627-0e70d94562f9 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:50.904266 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90f7cf-24fc-41f9-b266-00bd1a7f4cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:50.904916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90f7cf-24fc-41f9-b266-00bd1a7f4cd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:50.905198 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90f7cf-24fc-41f9-b266-00bd1a7f4cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:50.905473 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90f7cf-24fc-41f9-b266-00bd1a7f4cd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:50.906152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90f7cf-24fc-41f9-b266-00bd1a7f4cd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:50.908164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90f7cf-24fc-41f9-b266-00bd1a7f4cd8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:50.909450 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00514922-63bd-4366-a6e2-5fdc0853a747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:50.910327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00514922-63bd-4366-a6e2-5fdc0853a747 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:50.910826 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00514922-63bd-4366-a6e2-5fdc0853a747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:50.911324 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00514922-63bd-4366-a6e2-5fdc0853a747 admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:50.912770 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00514922-63bd-4366-a6e2-5fdc0853a747 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:59:50.913323 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00514922-63bd-4366-a6e2-5fdc0853a747 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:50.915898 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00514922-63bd-4366-a6e2-5fdc0853a747 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:50.932619 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 775/1568] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 04:59:50 2022] POST /identity/v3/users => generated 288 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:59:50.933263 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90f7cf-24fc-41f9-b266-00bd1a7f4cd8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:50.939905 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 794/1569] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 04:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:59:50.941416 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00514922-63bd-4366-a6e2-5fdc0853a747 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:50.943846 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90f7cf-24fc-41f9-b266-00bd1a7f4cd8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:50.948801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560346d-b59e-4f72-a8bd-6385d7fdb1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:50.949505 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560346d-b59e-4f72-a8bd-6385d7fdb1c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:50.949761 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560346d-b59e-4f72-a8bd-6385d7fdb1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:50.950003 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560346d-b59e-4f72-a8bd-6385d7fdb1c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:50.950678 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a560346d-b59e-4f72-a8bd-6385d7fdb1c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:50.952672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a560346d-b59e-4f72-a8bd-6385d7fdb1c6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:50.959818 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00514922-63bd-4366-a6e2-5fdc0853a747 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:50.968255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075ccde-d928-404f-bb4e-15f071f00a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:50.969447 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075ccde-d928-404f-bb4e-15f071f00a62 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:50.969997 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075ccde-d928-404f-bb4e-15f071f00a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:50.970480 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075ccde-d928-404f-bb4e-15f071f00a62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:50.971763 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075ccde-d928-404f-bb4e-15f071f00a62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:50.974923 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075ccde-d928-404f-bb4e-15f071f00a62 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:50.976206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 776/1570] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 04:59:50 2022] POST /identity/v3/users => generated 289 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:59:50.976838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560346d-b59e-4f72-a8bd-6385d7fdb1c6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:50.987731 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560346d-b59e-4f72-a8bd-6385d7fdb1c6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:50.993412 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba388355-016b-4299-a331-abde5140796a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:50.994730 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba388355-016b-4299-a331-abde5140796a admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:50.995306 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba388355-016b-4299-a331-abde5140796a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:50.995793 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba388355-016b-4299-a331-abde5140796a admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:50.996543 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba388355-016b-4299-a331-abde5140796a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:50.998221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba388355-016b-4299-a331-abde5140796a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:51.013158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 777/1571] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 04:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 04:59:51.014195 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba388355-016b-4299-a331-abde5140796a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:51.018699 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 795/1572] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 04:59:50 2022] POST /identity/v3/projects => generated 371 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:59:51.019924 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075ccde-d928-404f-bb4e-15f071f00a62 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:51.026907 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba388355-016b-4299-a331-abde5140796a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:51.066463 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:59:51.074234 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:59:51.110067 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1769d2-d8e8-4861-918b-cd7c649d6cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:51.110745 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1769d2-d8e8-4861-918b-cd7c649d6cc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:51.111032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1769d2-d8e8-4861-918b-cd7c649d6cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:51.111309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1769d2-d8e8-4861-918b-cd7c649d6cc0 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:51.112039 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1769d2-d8e8-4861-918b-cd7c649d6cc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:51.114058 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1769d2-d8e8-4861-918b-cd7c649d6cc0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:51.120309 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075ccde-d928-404f-bb4e-15f071f00a62 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:51.127440 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abadb9a2-d1f4-43ee-8abd-3cc71ce72796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:51.128197 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abadb9a2-d1f4-43ee-8abd-3cc71ce72796 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:51.128483 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abadb9a2-d1f4-43ee-8abd-3cc71ce72796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:51.128766 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abadb9a2-d1f4-43ee-8abd-3cc71ce72796 admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:51.129483 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abadb9a2-d1f4-43ee-8abd-3cc71ce72796 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:59:51.129858 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abadb9a2-d1f4-43ee-8abd-3cc71ce72796 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:51.131736 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abadb9a2-d1f4-43ee-8abd-3cc71ce72796 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:51.140516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 778/1573] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 04:59:51 2022] POST /identity/v3/users => generated 283 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:59:51.141457 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1769d2-d8e8-4861-918b-cd7c649d6cc0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:51.150367 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 796/1574] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 04:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:59:51.151375 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abadb9a2-d1f4-43ee-8abd-3cc71ce72796 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:51.152143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1769d2-d8e8-4861-918b-cd7c649d6cc0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:51.161394 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c84b6-531b-4cf2-b5a6-4e9763bca822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:51.162147 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c84b6-531b-4cf2-b5a6-4e9763bca822 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:51.162440 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c84b6-531b-4cf2-b5a6-4e9763bca822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:51.162718 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c84b6-531b-4cf2-b5a6-4e9763bca822 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:51.163408 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60c84b6-531b-4cf2-b5a6-4e9763bca822 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:51.165633 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60c84b6-531b-4cf2-b5a6-4e9763bca822 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:51.166884 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abadb9a2-d1f4-43ee-8abd-3cc71ce72796 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:51.175009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24cd0ba-6621-4829-9550-40b35f62adbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:51.175949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24cd0ba-6621-4829-9550-40b35f62adbe admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:51.176302 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24cd0ba-6621-4829-9550-40b35f62adbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:51.176549 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24cd0ba-6621-4829-9550-40b35f62adbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:51.177319 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24cd0ba-6621-4829-9550-40b35f62adbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:51.179080 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24cd0ba-6621-4829-9550-40b35f62adbe admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:51.191700 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 797/1575] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 04:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 04:59:51.192477 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24cd0ba-6621-4829-9550-40b35f62adbe admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:51.204161 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24cd0ba-6621-4829-9550-40b35f62adbe admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:51.206092 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 779/1576] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 04:59:51 2022] POST /identity/v3/projects => generated 377 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:59:51.210140 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c84b6-531b-4cf2-b5a6-4e9763bca822 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:51.275251 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ed842-f2f1-41c1-9978-9c34a6017a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:51.275994 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ed842-f2f1-41c1-9978-9c34a6017a31 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:51.276278 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ed842-f2f1-41c1-9978-9c34a6017a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:51.276559 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ed842-f2f1-41c1-9978-9c34a6017a31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:51.277287 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75ed842-f2f1-41c1-9978-9c34a6017a31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:51.279313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75ed842-f2f1-41c1-9978-9c34a6017a31 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:51.283625 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c84b6-531b-4cf2-b5a6-4e9763bca822 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:51.288877 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7206c-3919-4505-b402-4a54fd744c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:51.289304 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7206c-3919-4505-b402-4a54fd744c95 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:51.289570 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7206c-3919-4505-b402-4a54fd744c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:51.289849 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7206c-3919-4505-b402-4a54fd744c95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:51.290721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd7206c-3919-4505-b402-4a54fd744c95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:51.292818 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd7206c-3919-4505-b402-4a54fd744c95 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:51.308502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 780/1577] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 04:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 04:59:51.309636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7206c-3919-4505-b402-4a54fd744c95 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:51.321488 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7206c-3919-4505-b402-4a54fd744c95 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:51.323158 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 798/1578] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 04:59:51 2022] POST /identity/v3/projects => generated 391 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:59:51.324411 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ed842-f2f1-41c1-9978-9c34a6017a31 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:51.398068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ea9e0-c18b-4441-9636-1a3148d4a0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:51.398818 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ea9e0-c18b-4441-9636-1a3148d4a0a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:51.399094 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ea9e0-c18b-4441-9636-1a3148d4a0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:51.399372 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ea9e0-c18b-4441-9636-1a3148d4a0a7 admin admin] PATH_INFO: `/v3/projects/b8af7edda5f24572bdbf92e950c73ff5/users/55c6c3c7738e477eb1edc784967559ee/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:51.413387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ed842-f2f1-41c1-9978-9c34a6017a31 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:51.417055 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ea9e0-c18b-4441-9636-1a3148d4a0a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8af7edda5f24572bdbf92e950c73ff5, user_id=55c6c3c7738e477eb1edc784967559ee, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:51.418131 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-9f6ea9e0-c18b-4441-9636-1a3148d4a0a7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 04:59:51.418131 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jul 26 04:59:51.418131 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: Jul 26 04:59:51.418746 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69591b-912c-4ef1-bab7-3d7394df13de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:51.419399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69591b-912c-4ef1-bab7-3d7394df13de admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:51.419704 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69591b-912c-4ef1-bab7-3d7394df13de admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:51.419971 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69591b-912c-4ef1-bab7-3d7394df13de admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:51.420160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ea9e0-c18b-4441-9636-1a3148d4a0a7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:51.420648 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e69591b-912c-4ef1-bab7-3d7394df13de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:51.422651 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e69591b-912c-4ef1-bab7-3d7394df13de admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:51.437926 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 781/1579] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 04:59:51 2022] PUT /identity/v3/projects/b8af7edda5f24572bdbf92e950c73ff5/users/55c6c3c7738e477eb1edc784967559ee/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:59:51.447707 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ea9e0-c18b-4441-9636-1a3148d4a0a7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:51.447890 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 799/1580] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 04:59:51 2022] POST /identity/v3/users => generated 286 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:59:51.450675 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69591b-912c-4ef1-bab7-3d7394df13de admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:51.464946 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69591b-912c-4ef1-bab7-3d7394df13de admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:51.478691 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:59:51.496477 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:59:51.529559 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ea9e0-c18b-4441-9636-1a3148d4a0a7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:51.534645 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b20746-ad0f-4d25-a42f-461ba19cbdb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:51.535308 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b20746-ad0f-4d25-a42f-461ba19cbdb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:51.536228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f83c8-c0b4-4f77-ab88-6c0e691301f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:51.536357 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b20746-ad0f-4d25-a42f-461ba19cbdb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:51.536357 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b20746-ad0f-4d25-a42f-461ba19cbdb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:51.537307 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f83c8-c0b4-4f77-ab88-6c0e691301f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:51.537307 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f83c8-c0b4-4f77-ab88-6c0e691301f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:51.537510 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b20746-ad0f-4d25-a42f-461ba19cbdb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:51.537610 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f83c8-c0b4-4f77-ab88-6c0e691301f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:51.537894 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f83c8-c0b4-4f77-ab88-6c0e691301f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:51.538434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b20746-ad0f-4d25-a42f-461ba19cbdb0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:51.540430 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f83c8-c0b4-4f77-ab88-6c0e691301f8 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:51.550351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 782/1581] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 04:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 04:59:51.551279 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b20746-ad0f-4d25-a42f-461ba19cbdb0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:51.562509 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b20746-ad0f-4d25-a42f-461ba19cbdb0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:51.569314 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623851e5-5a84-4ee9-8b5e-d8dfc8f3d33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:51.569991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623851e5-5a84-4ee9-8b5e-d8dfc8f3d33e admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:51.570322 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623851e5-5a84-4ee9-8b5e-d8dfc8f3d33e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:51.570736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623851e5-5a84-4ee9-8b5e-d8dfc8f3d33e admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:51.571481 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623851e5-5a84-4ee9-8b5e-d8dfc8f3d33e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:51.573268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623851e5-5a84-4ee9-8b5e-d8dfc8f3d33e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:51.582775 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 800/1582] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 04:59:51 2022] POST /identity/v3/projects => generated 389 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:59:51.582775 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f83c8-c0b4-4f77-ab88-6c0e691301f8 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:51.597699 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 783/1583] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 04:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 04:59:51.597699 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623851e5-5a84-4ee9-8b5e-d8dfc8f3d33e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:51.612329 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623851e5-5a84-4ee9-8b5e-d8dfc8f3d33e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:51.685305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74904202-bcdf-47f4-83cd-2be4032ad6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:51.686041 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74904202-bcdf-47f4-83cd-2be4032ad6d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:51.686345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74904202-bcdf-47f4-83cd-2be4032ad6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:51.686661 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74904202-bcdf-47f4-83cd-2be4032ad6d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:51.687392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74904202-bcdf-47f4-83cd-2be4032ad6d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:51.689126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74904202-bcdf-47f4-83cd-2be4032ad6d3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:51.697619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f83c8-c0b4-4f77-ab88-6c0e691301f8 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:51.700596 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 784/1584] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 04:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 04:59:51.701489 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74904202-bcdf-47f4-83cd-2be4032ad6d3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:51.703099 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed236ad-d343-424c-a988-a7cba2d09d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:51.705141 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed236ad-d343-424c-a988-a7cba2d09d1b admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:51.705427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed236ad-d343-424c-a988-a7cba2d09d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:51.705701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed236ad-d343-424c-a988-a7cba2d09d1b admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:51.706445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed236ad-d343-424c-a988-a7cba2d09d1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:51.708441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed236ad-d343-424c-a988-a7cba2d09d1b admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:51.712261 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74904202-bcdf-47f4-83cd-2be4032ad6d3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:51.717270 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81808da-71ae-4fd4-bf7e-695e28d89bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:51.717962 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81808da-71ae-4fd4-bf7e-695e28d89bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:51.718368 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81808da-71ae-4fd4-bf7e-695e28d89bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:51.718654 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81808da-71ae-4fd4-bf7e-695e28d89bb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:51.719577 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81808da-71ae-4fd4-bf7e-695e28d89bb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:51.722135 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81808da-71ae-4fd4-bf7e-695e28d89bb4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:51.732827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 801/1585] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 04:59:51 2022] POST /identity/v3/users => generated 289 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:59:51.734009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07779934-cdf5-408d-8c1b-47cd2c56b550 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:51.734307 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07779934-cdf5-408d-8c1b-47cd2c56b550 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:51.734557 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07779934-cdf5-408d-8c1b-47cd2c56b550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:51.755737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 785/1586] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 04:59:51 2022] POST /identity/v3/users => generated 296 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:59:51.756886 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81808da-71ae-4fd4-bf7e-695e28d89bb4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:51.767906 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81808da-71ae-4fd4-bf7e-695e28d89bb4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:51.769511 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-07779934-cdf5-408d-8c1b-47cd2c56b550 None None] MFA Rules not processed for user `55c6c3c7738e477eb1edc784967559ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:51.770789 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-07779934-cdf5-408d-8c1b-47cd2c56b550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:51.774603 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ec1c3-3447-42d1-b649-248f31af1b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:51.775413 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ec1c3-3447-42d1-b649-248f31af1b39 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:51.775884 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ec1c3-3447-42d1-b649-248f31af1b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:51.776229 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ec1c3-3447-42d1-b649-248f31af1b39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:51.777191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37ec1c3-3447-42d1-b649-248f31af1b39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:51.779602 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37ec1c3-3447-42d1-b649-248f31af1b39 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:51.793068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 786/1587] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 04:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 04:59:51.794128 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ec1c3-3447-42d1-b649-248f31af1b39 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:51.806711 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ec1c3-3447-42d1-b649-248f31af1b39 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:51.812840 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e4a55-df5f-4a20-9c03-61ed4f942377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:51.813530 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e4a55-df5f-4a20-9c03-61ed4f942377 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:51.813820 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e4a55-df5f-4a20-9c03-61ed4f942377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:51.814101 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e4a55-df5f-4a20-9c03-61ed4f942377 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:51.814812 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e4a55-df5f-4a20-9c03-61ed4f942377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:51.816494 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e4a55-df5f-4a20-9c03-61ed4f942377 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:51.828551 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 787/1588] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 04:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 04:59:51.829691 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e4a55-df5f-4a20-9c03-61ed4f942377 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:51.841507 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e4a55-df5f-4a20-9c03-61ed4f942377 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:51.846954 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3afcd-61de-4b47-b404-78ecfe594a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:51.847630 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3afcd-61de-4b47-b404-78ecfe594a28 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:51.847916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3afcd-61de-4b47-b404-78ecfe594a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:51.848198 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3afcd-61de-4b47-b404-78ecfe594a28 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:51.848891 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba3afcd-61de-4b47-b404-78ecfe594a28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:51.850992 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba3afcd-61de-4b47-b404-78ecfe594a28 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:51.881696 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 788/1589] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 04:59:51 2022] POST /identity/v3/users => generated 295 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:59:51.881696 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3afcd-61de-4b47-b404-78ecfe594a28 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:51.888054 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 802/1590] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:51 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:51.888924 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07779934-cdf5-408d-8c1b-47cd2c56b550 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:51.889818 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3afcd-61de-4b47-b404-78ecfe594a28 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:51.894833 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c0356-606a-4442-be75-0a7c50a96af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:51.895637 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c0356-606a-4442-be75-0a7c50a96af1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:51.895866 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c0356-606a-4442-be75-0a7c50a96af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:51.896149 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c0356-606a-4442-be75-0a7c50a96af1 admin admin] PATH_INFO: `/v3/projects/8653b1f3efea450dae2a0881cb757c86/users/92a01d413b71400da5885d4ea44ec03a/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:51.903400 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07779934-cdf5-408d-8c1b-47cd2c56b550 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:51.908183 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbcfbcd-6164-4c3b-9e38-e69adb4487b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:51.908899 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcfbcd-6164-4c3b-9e38-e69adb4487b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:51.909186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcfbcd-6164-4c3b-9e38-e69adb4487b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:51.909472 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcfbcd-6164-4c3b-9e38-e69adb4487b3 admin admin] PATH_INFO: `/v3/projects/9747c4b81e8844e989d70b8001a196b0/users/39de5897bd2d4e2b91c05d8dda5827be/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:51.921326 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448c0356-606a-4442-be75-0a7c50a96af1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8653b1f3efea450dae2a0881cb757c86, user_id=92a01d413b71400da5885d4ea44ec03a, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:51.926614 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbcfbcd-6164-4c3b-9e38-e69adb4487b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9747c4b81e8844e989d70b8001a196b0, user_id=39de5897bd2d4e2b91c05d8dda5827be, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:51.926876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448c0356-606a-4442-be75-0a7c50a96af1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:51.929479 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbcfbcd-6164-4c3b-9e38-e69adb4487b3 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:51.945556 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 789/1591] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 04:59:51 2022] PUT /identity/v3/projects/8653b1f3efea450dae2a0881cb757c86/users/92a01d413b71400da5885d4ea44ec03a/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:59:51.946442 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c0356-606a-4442-be75-0a7c50a96af1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:51.955258 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 803/1592] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 04:59:51 2022] PUT /identity/v3/projects/9747c4b81e8844e989d70b8001a196b0/users/39de5897bd2d4e2b91c05d8dda5827be/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:59:51.955258 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbcfbcd-6164-4c3b-9e38-e69adb4487b3 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:51.970756 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:59:51.987774 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:59:52.040164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c0356-606a-4442-be75-0a7c50a96af1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:52.052322 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbcfbcd-6164-4c3b-9e38-e69adb4487b3 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:52.057475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8d13c-e53b-4d95-adb5-0bbdd7e2ad6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:52.058131 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8d13c-e53b-4d95-adb5-0bbdd7e2ad6a admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:52.058422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8d13c-e53b-4d95-adb5-0bbdd7e2ad6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:52.058895 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8d13c-e53b-4d95-adb5-0bbdd7e2ad6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:52.059663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8d13c-e53b-4d95-adb5-0bbdd7e2ad6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:52.061572 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8d13c-e53b-4d95-adb5-0bbdd7e2ad6a admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:52.074346 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 804/1593] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 04:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 04:59:52.075261 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8d13c-e53b-4d95-adb5-0bbdd7e2ad6a admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:52.087787 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8d13c-e53b-4d95-adb5-0bbdd7e2ad6a admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:52.093702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813166c9-9985-4dff-b833-05e727ee4f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:52.094359 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813166c9-9985-4dff-b833-05e727ee4f3a admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:52.094645 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813166c9-9985-4dff-b833-05e727ee4f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:52.094923 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813166c9-9985-4dff-b833-05e727ee4f3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:52.095688 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813166c9-9985-4dff-b833-05e727ee4f3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:52.097427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813166c9-9985-4dff-b833-05e727ee4f3a admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:52.109612 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 805/1594] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 04:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 04:59:52.110513 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813166c9-9985-4dff-b833-05e727ee4f3a admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:52.120904 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813166c9-9985-4dff-b833-05e727ee4f3a admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:52.126516 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf5488c-430e-4e42-ac42-1cfe42d69e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:52.127275 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5488c-430e-4e42-ac42-1cfe42d69e62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:52.127501 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5488c-430e-4e42-ac42-1cfe42d69e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:52.127823 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5488c-430e-4e42-ac42-1cfe42d69e62 admin admin] PATH_INFO: `/v3/projects/c295b736057e4f73b562a5c41f9717dd/users/44065be4a0c541289827f5586aa43c69/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:52.129457 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b1f2e9-f01c-4803-aed2-a65baf1a1044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:52.130122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1f2e9-f01c-4803-aed2-a65baf1a1044 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:52.130790 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1f2e9-f01c-4803-aed2-a65baf1a1044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:52.131066 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1f2e9-f01c-4803-aed2-a65baf1a1044 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:52.131824 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b1f2e9-f01c-4803-aed2-a65baf1a1044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:52.133460 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b1f2e9-f01c-4803-aed2-a65baf1a1044 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:52.144426 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 790/1595] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 04:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 04:59:52.145379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b1f2e9-f01c-4803-aed2-a65baf1a1044 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:52.147406 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf5488c-430e-4e42-ac42-1cfe42d69e62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c295b736057e4f73b562a5c41f9717dd, user_id=44065be4a0c541289827f5586aa43c69, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:52.150277 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf5488c-430e-4e42-ac42-1cfe42d69e62 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:52.157976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b1f2e9-f01c-4803-aed2-a65baf1a1044 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:52.163718 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae8938c-caea-419e-b576-fafc117cf878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:52.164373 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8938c-caea-419e-b576-fafc117cf878 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:52.164660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8938c-caea-419e-b576-fafc117cf878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:52.164936 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8938c-caea-419e-b576-fafc117cf878 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:52.165639 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae8938c-caea-419e-b576-fafc117cf878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:52.167278 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae8938c-caea-419e-b576-fafc117cf878 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:52.169181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 806/1596] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 04:59:52 2022] PUT /identity/v3/projects/c295b736057e4f73b562a5c41f9717dd/users/44065be4a0c541289827f5586aa43c69/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:59:52.170691 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf5488c-430e-4e42-ac42-1cfe42d69e62 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:52.183336 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 791/1597] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 04:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 04:59:52.183336 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2ef3f-58f9-4a0f-832f-f0841f8a1725 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:52.183336 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2ef3f-58f9-4a0f-832f-f0841f8a1725 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:52.183336 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2ef3f-58f9-4a0f-832f-f0841f8a1725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:52.204676 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0dc2ef3f-58f9-4a0f-832f-f0841f8a1725 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:52.205981 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0dc2ef3f-58f9-4a0f-832f-f0841f8a1725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:52.261438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf5488c-430e-4e42-ac42-1cfe42d69e62 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:52.266074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e3e32-7355-41a3-8754-89eced5c0ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:52.266705 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e3e32-7355-41a3-8754-89eced5c0ce2 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:52.266986 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e3e32-7355-41a3-8754-89eced5c0ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:52.267257 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e3e32-7355-41a3-8754-89eced5c0ce2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:52.267969 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e3e32-7355-41a3-8754-89eced5c0ce2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:52.269908 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e3e32-7355-41a3-8754-89eced5c0ce2 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:52.275788 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 792/1598] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:52.277295 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe191f3-6274-46d3-80a4-6ee0c966469a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:52.277752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe191f3-6274-46d3-80a4-6ee0c966469a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:52.278112 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe191f3-6274-46d3-80a4-6ee0c966469a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:52.309453 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 807/1599] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 04:59:52 2022] POST /identity/v3/projects => generated 405 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:59:52.310530 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e3e32-7355-41a3-8754-89eced5c0ce2 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:52.316777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ebe191f3-6274-46d3-80a4-6ee0c966469a None None] MFA Rules not processed for user `92a01d413b71400da5885d4ea44ec03a`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:52.318024 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ebe191f3-6274-46d3-80a4-6ee0c966469a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:52.372875 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:59:52.394990 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:59:52.397802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 793/1600] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:52.398961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe191f3-6274-46d3-80a4-6ee0c966469a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:52.403398 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e3e32-7355-41a3-8754-89eced5c0ce2 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:52.410979 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69efe73-dce4-4b52-bbd2-fdbb5a0a1007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:52.412329 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69efe73-dce4-4b52-bbd2-fdbb5a0a1007 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:52.412329 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69efe73-dce4-4b52-bbd2-fdbb5a0a1007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:52.412559 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69efe73-dce4-4b52-bbd2-fdbb5a0a1007 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:52.413271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69efe73-dce4-4b52-bbd2-fdbb5a0a1007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:52.414712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe191f3-6274-46d3-80a4-6ee0c966469a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:52.415025 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69efe73-dce4-4b52-bbd2-fdbb5a0a1007 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:52.424865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b6e89-32b1-485a-a5d9-a9f3725b1d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:52.426080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b6e89-32b1-485a-a5d9-a9f3725b1d92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:52.426621 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b6e89-32b1-485a-a5d9-a9f3725b1d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:52.427107 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b6e89-32b1-485a-a5d9-a9f3725b1d92 admin admin] PATH_INFO: `/v3/projects/f797e5a15e8b4bc887794be8dc622961/users/c84bbfe30cb04dd5adcd67987e31412d/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:52.433612 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 808/1601] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 04:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 04:59:52.433612 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69efe73-dce4-4b52-bbd2-fdbb5a0a1007 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:52.447925 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69efe73-dce4-4b52-bbd2-fdbb5a0a1007 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:52.453533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7478b2-0dbb-49f9-b9a7-037a7fd93900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:52.454188 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7478b2-0dbb-49f9-b9a7-037a7fd93900 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:52.454465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7478b2-0dbb-49f9-b9a7-037a7fd93900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:52.454735 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7478b2-0dbb-49f9-b9a7-037a7fd93900 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:52.455627 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7478b2-0dbb-49f9-b9a7-037a7fd93900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:52.457777 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7478b2-0dbb-49f9-b9a7-037a7fd93900 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:52.459048 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b6e89-32b1-485a-a5d9-a9f3725b1d92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f797e5a15e8b4bc887794be8dc622961, user_id=c84bbfe30cb04dd5adcd67987e31412d, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:52.462076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b6e89-32b1-485a-a5d9-a9f3725b1d92 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:52.471540 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 809/1602] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 04:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 04:59:52.472452 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7478b2-0dbb-49f9-b9a7-037a7fd93900 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:52.481971 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 794/1603] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 04:59:52 2022] PUT /identity/v3/projects/f797e5a15e8b4bc887794be8dc622961/users/c84bbfe30cb04dd5adcd67987e31412d/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:59:52.483610 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7478b2-0dbb-49f9-b9a7-037a7fd93900 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:52.484581 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4aad1c-09d5-4325-9d92-7ca6b64978f5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:52.485151 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4aad1c-09d5-4325-9d92-7ca6b64978f5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:52.485710 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4aad1c-09d5-4325-9d92-7ca6b64978f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:52.531602 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1a4aad1c-09d5-4325-9d92-7ca6b64978f5 None None] MFA Rules not processed for user `44065be4a0c541289827f5586aa43c69`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:52.532882 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1a4aad1c-09d5-4325-9d92-7ca6b64978f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:52.565575 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35f227-57c9-4ec4-af95-2e9d150ba8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:52.566633 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35f227-57c9-4ec4-af95-2e9d150ba8d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:52.566633 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35f227-57c9-4ec4-af95-2e9d150ba8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:52.566816 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35f227-57c9-4ec4-af95-2e9d150ba8d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:52.567809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c35f227-57c9-4ec4-af95-2e9d150ba8d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:52.569834 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c35f227-57c9-4ec4-af95-2e9d150ba8d9 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:52.582332 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 810/1604] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 04:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 04:59:52.583207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35f227-57c9-4ec4-af95-2e9d150ba8d9 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:52.612538 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 795/1605] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:52 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:52.613433 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4aad1c-09d5-4325-9d92-7ca6b64978f5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:52.624990 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4aad1c-09d5-4325-9d92-7ca6b64978f5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:52.629610 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae5c9f9-fdd6-4e7e-aaec-3205633e5ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:52.630241 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5c9f9-fdd6-4e7e-aaec-3205633e5ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:52.630519 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5c9f9-fdd6-4e7e-aaec-3205633e5ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:52.630946 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5c9f9-fdd6-4e7e-aaec-3205633e5ee3 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:52.631668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae5c9f9-fdd6-4e7e-aaec-3205633e5ee3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:52.633606 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae5c9f9-fdd6-4e7e-aaec-3205633e5ee3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:52.648357 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35f227-57c9-4ec4-af95-2e9d150ba8d9 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:52.652265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7beb5f3-174d-422d-9c91-d6eedb86b957 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:52.652912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7beb5f3-174d-422d-9c91-d6eedb86b957 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:52.653198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7beb5f3-174d-422d-9c91-d6eedb86b957 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:52.653441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7beb5f3-174d-422d-9c91-d6eedb86b957 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:52.660600 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 796/1606] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 04:59:52 2022] POST /identity/v3/users => generated 309 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:59:52.660600 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21544496-c3a4-45fb-976f-c976680ec850 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:52.660600 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21544496-c3a4-45fb-976f-c976680ec850 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:52.660600 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21544496-c3a4-45fb-976f-c976680ec850 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:52.660600 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 797/1607] 10.222.0.17 () {54 vars in 962 bytes} [Tue Jul 26 04:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:59:52.661118 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21544496-c3a4-45fb-976f-c976680ec850 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:52.663950 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7beb5f3-174d-422d-9c91-d6eedb86b957 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:52.665403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7beb5f3-174d-422d-9c91-d6eedb86b957 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:52.674033 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21544496-c3a4-45fb-976f-c976680ec850 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:52.675629 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 811/1608] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:59:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:59:52.676351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7beb5f3-174d-422d-9c91-d6eedb86b957 service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:52.679106 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c59485-9a57-48fb-9e02-a9ace5651600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:52.679941 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c59485-9a57-48fb-9e02-a9ace5651600 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:52.680378 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c59485-9a57-48fb-9e02-a9ace5651600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:52.680874 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c59485-9a57-48fb-9e02-a9ace5651600 admin admin] PATH_INFO: `/v3/projects/1c772e5c90114978ad92818d58ca11c2/users/f0cc1af7d7d642889a1a11103f087e79/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:52.687668 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7beb5f3-174d-422d-9c91-d6eedb86b957 service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:52.693125 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91724322-1289-4e8e-83fc-2d6df337ac25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:52.693872 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91724322-1289-4e8e-83fc-2d6df337ac25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:52.694154 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91724322-1289-4e8e-83fc-2d6df337ac25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:52.694433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91724322-1289-4e8e-83fc-2d6df337ac25 admin admin] PATH_INFO: `/v3/projects/981b76b8fd394154a07921073ab08ed6/users/f4e86ac513aa4feda193e134c0bc9e79/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:52.701754 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c59485-9a57-48fb-9e02-a9ace5651600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c772e5c90114978ad92818d58ca11c2, user_id=f0cc1af7d7d642889a1a11103f087e79, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:52.705127 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c59485-9a57-48fb-9e02-a9ace5651600 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:52.713808 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91724322-1289-4e8e-83fc-2d6df337ac25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=981b76b8fd394154a07921073ab08ed6, user_id=f4e86ac513aa4feda193e134c0bc9e79, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:52.717196 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91724322-1289-4e8e-83fc-2d6df337ac25 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:52.720156 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 798/1609] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 04:59:52 2022] PUT /identity/v3/projects/1c772e5c90114978ad92818d58ca11c2/users/f0cc1af7d7d642889a1a11103f087e79/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:59:52.726212 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c2226-2ef2-49f2-b5d2-24dae135adaf None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:52.726212 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c2226-2ef2-49f2-b5d2-24dae135adaf None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:52.726212 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c2226-2ef2-49f2-b5d2-24dae135adaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:52.737412 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 812/1610] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 04:59:52 2022] PUT /identity/v3/projects/981b76b8fd394154a07921073ab08ed6/users/f4e86ac513aa4feda193e134c0bc9e79/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:59:52.738099 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91724322-1289-4e8e-83fc-2d6df337ac25 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:52.757445 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-831c2226-2ef2-49f2-b5d2-24dae135adaf None None] MFA Rules not processed for user `c84bbfe30cb04dd5adcd67987e31412d`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:52.758984 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-831c2226-2ef2-49f2-b5d2-24dae135adaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:52.802065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:59:52.808745 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91724322-1289-4e8e-83fc-2d6df337ac25 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:52.814771 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72b750-be9c-4594-8473-854a949fd377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:52.816034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72b750-be9c-4594-8473-854a949fd377 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:52.816320 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72b750-be9c-4594-8473-854a949fd377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:52.816665 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72b750-be9c-4594-8473-854a949fd377 admin admin] PATH_INFO: `/v3/projects/27187d178a5f47b49035991ab566c2d9/users/f996bed3f6064c6dbf04d26c7c04df7c/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:52.830084 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:59:52.834342 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a72b750-be9c-4594-8473-854a949fd377 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27187d178a5f47b49035991ab566c2d9, user_id=f996bed3f6064c6dbf04d26c7c04df7c, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:52.840549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 799/1611] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:52.840549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c2226-2ef2-49f2-b5d2-24dae135adaf None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:52.840824 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a72b750-be9c-4594-8473-854a949fd377 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:52.884193 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 813/1612] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 04:59:52 2022] PUT /identity/v3/projects/27187d178a5f47b49035991ab566c2d9/users/f996bed3f6064c6dbf04d26c7c04df7c/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:59:52.885173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72b750-be9c-4594-8473-854a949fd377 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:52.925944 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c2226-2ef2-49f2-b5d2-24dae135adaf None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:52.965332 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72b750-be9c-4594-8473-854a949fd377 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:52.973118 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85804d7b-8684-47ad-9b5c-49b82008e596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:52.973914 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85804d7b-8684-47ad-9b5c-49b82008e596 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:52.974238 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85804d7b-8684-47ad-9b5c-49b82008e596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:52.974650 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85804d7b-8684-47ad-9b5c-49b82008e596 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:52.975452 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85804d7b-8684-47ad-9b5c-49b82008e596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:52.977333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85804d7b-8684-47ad-9b5c-49b82008e596 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:52.988206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25621af6-d054-43c5-be0b-9928fb4e931d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:52.992169 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25621af6-d054-43c5-be0b-9928fb4e931d service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:52.992334 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 814/1613] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 04:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 04:59:52.992443 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25621af6-d054-43c5-be0b-9928fb4e931d service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:52.992443 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25621af6-d054-43c5-be0b-9928fb4e931d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:52.992617 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c9040-1435-4eae-943e-92840ed6bd3a None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:52.992617 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c9040-1435-4eae-943e-92840ed6bd3a None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:52.992617 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c9040-1435-4eae-943e-92840ed6bd3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:53.036666 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-2d6c9040-1435-4eae-943e-92840ed6bd3a None None] MFA Rules not processed for user `2faeed9002ee4154bc980c146cd06bcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:53.037971 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-2d6c9040-1435-4eae-943e-92840ed6bd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:53.064565 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25621af6-d054-43c5-be0b-9928fb4e931d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:53.066497 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25621af6-d054-43c5-be0b-9928fb4e931d service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:53.083542 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 800/1614] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:59:52 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:59:53.086008 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28074ff5-6860-4415-89ab-bc95331f66da None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:53.086715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28074ff5-6860-4415-89ab-bc95331f66da None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:53.087216 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28074ff5-6860-4415-89ab-bc95331f66da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:53.112188 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-28074ff5-6860-4415-89ab-bc95331f66da None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:53.113146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-28074ff5-6860-4415-89ab-bc95331f66da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:53.123133 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 801/1615] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:53.125438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2f4be-547b-4ad7-8f45-abb0444af6f3 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:53.125724 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2f4be-547b-4ad7-8f45-abb0444af6f3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:53.132992 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2f4be-547b-4ad7-8f45-abb0444af6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:53.145416 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 815/1616] 10.222.0.17 () {58 vars in 1059 bytes} [Tue Jul 26 04:59:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:53.152778 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-28a2f4be-547b-4ad7-8f45-abb0444af6f3 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:53.154354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-28a2f4be-547b-4ad7-8f45-abb0444af6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:53.155350 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0c5e3-7ec0-4510-bcba-f8226fa2b1ae None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:53.155791 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0c5e3-7ec0-4510-bcba-f8226fa2b1ae None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:53.156234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0c5e3-7ec0-4510-bcba-f8226fa2b1ae None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:53.160102 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 816/1617] 10.222.0.17 () {54 vars in 962 bytes} [Tue Jul 26 04:59:53 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:59:53.163689 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 802/1618] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:53.164537 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2f4be-547b-4ad7-8f45-abb0444af6f3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:53.173172 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20622a05-d674-4209-8369-d78a817b9d53 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:53.173579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20622a05-d674-4209-8369-d78a817b9d53 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:53.173959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20622a05-d674-4209-8369-d78a817b9d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:53.174544 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2f4be-547b-4ad7-8f45-abb0444af6f3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:53.179448 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fa6b2-61e1-4368-98b0-7ff141cbcef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:53.180167 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fa6b2-61e1-4368-98b0-7ff141cbcef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:53.180453 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fa6b2-61e1-4368-98b0-7ff141cbcef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:53.180731 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fa6b2-61e1-4368-98b0-7ff141cbcef4 admin admin] PATH_INFO: `/v3/projects/49bc5f90455e43cdafc1d2b452bd4c9f/users/700b9fc94b0a4e5f99e86fbe4410b0fe/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:53.198293 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65fa6b2-61e1-4368-98b0-7ff141cbcef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49bc5f90455e43cdafc1d2b452bd4c9f, user_id=700b9fc94b0a4e5f99e86fbe4410b0fe, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:53.201199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65fa6b2-61e1-4368-98b0-7ff141cbcef4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:53.204632 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-20622a05-d674-4209-8369-d78a817b9d53 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:53.206029 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-20622a05-d674-4209-8369-d78a817b9d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:53.215569 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 817/1619] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:53.216796 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20622a05-d674-4209-8369-d78a817b9d53 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:53.228247 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 803/1620] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 04:59:53 2022] PUT /identity/v3/projects/49bc5f90455e43cdafc1d2b452bd4c9f/users/700b9fc94b0a4e5f99e86fbe4410b0fe/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:59:53.229416 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fa6b2-61e1-4368-98b0-7ff141cbcef4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:53.276263 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:59:53.290112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:59:53.304189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20622a05-d674-4209-8369-d78a817b9d53 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:53.329019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fa6b2-61e1-4368-98b0-7ff141cbcef4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:53.338921 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c1514-c4ba-472d-8c81-3f2cf4112a4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:53.339994 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c1514-c4ba-472d-8c81-3f2cf4112a4b service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:53.340375 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c1514-c4ba-472d-8c81-3f2cf4112a4b service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:53.343119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c1514-c4ba-472d-8c81-3f2cf4112a4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:53.378172 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b76eb5-9306-4b4f-86fa-bd55a835aef4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:53.379271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b76eb5-9306-4b4f-86fa-bd55a835aef4 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:53.379834 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b76eb5-9306-4b4f-86fa-bd55a835aef4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:53.380411 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b76eb5-9306-4b4f-86fa-bd55a835aef4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:53.438765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c1514-c4ba-472d-8c81-3f2cf4112a4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:53.442925 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c1514-c4ba-472d-8c81-3f2cf4112a4b service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:53.455307 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 804/1621] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:59:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:59:53.470432 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3fb73-71a1-4f91-836f-445b77d52142 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:53.470739 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3fb73-71a1-4f91-836f-445b77d52142 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:53.471023 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3fb73-71a1-4f91-836f-445b77d52142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:53.502766 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b76eb5-9306-4b4f-86fa-bd55a835aef4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:53.507963 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b76eb5-9306-4b4f-86fa-bd55a835aef4 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:53.518597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-cca3fb73-71a1-4f91-836f-445b77d52142 None None] MFA Rules not processed for user `2faeed9002ee4154bc980c146cd06bcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:53.519613 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cca3fb73-71a1-4f91-836f-445b77d52142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:53.532730 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 818/1622] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:59:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:59:53.535105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b76eb5-9306-4b4f-86fa-bd55a835aef4 service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:53.542018 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 805/1623] 10.222.0.17 () {58 vars in 1059 bytes} [Tue Jul 26 04:59:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:53.544267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca3fb73-71a1-4f91-836f-445b77d52142 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:53.557840 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b76eb5-9306-4b4f-86fa-bd55a835aef4 service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:53.567986 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc838d56-fb20-420e-a29a-9ea007af4f7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:53.569143 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc838d56-fb20-420e-a29a-9ea007af4f7d service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:53.570076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca3fb73-71a1-4f91-836f-445b77d52142 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:53.570607 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc838d56-fb20-420e-a29a-9ea007af4f7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:53.571072 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc838d56-fb20-420e-a29a-9ea007af4f7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:53.585810 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a028da1-996d-4705-8c02-7d2bf49557a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:53.586320 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc838d56-fb20-420e-a29a-9ea007af4f7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:53.587956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc838d56-fb20-420e-a29a-9ea007af4f7d service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:53.594307 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a028da1-996d-4705-8c02-7d2bf49557a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:53.598069 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a028da1-996d-4705-8c02-7d2bf49557a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:53.599089 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a028da1-996d-4705-8c02-7d2bf49557a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:53.601306 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 819/1624] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:59:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:59:53.602380 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc838d56-fb20-420e-a29a-9ea007af4f7d service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:53.614544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc838d56-fb20-420e-a29a-9ea007af4f7d service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:53.623311 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a028da1-996d-4705-8c02-7d2bf49557a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:53.624416 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d329d4-6a91-4f75-836a-f555327e9889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:53.625529 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d329d4-6a91-4f75-836a-f555327e9889 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:53.626967 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d329d4-6a91-4f75-836a-f555327e9889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:53.627385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d329d4-6a91-4f75-836a-f555327e9889 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:53.628666 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d329d4-6a91-4f75-836a-f555327e9889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:53.628862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a028da1-996d-4705-8c02-7d2bf49557a0 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:53.632112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d329d4-6a91-4f75-836a-f555327e9889 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:53.651800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 820/1625] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 04:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 04:59:53.655476 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d329d4-6a91-4f75-836a-f555327e9889 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:53.660766 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 806/1626] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:59:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:59:53.661994 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a028da1-996d-4705-8c02-7d2bf49557a0 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:53.682866 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a028da1-996d-4705-8c02-7d2bf49557a0 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:53.696349 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d329d4-6a91-4f75-836a-f555327e9889 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:53.699471 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da6d82e-2830-4bd6-af9e-c81db8db51e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:53.700598 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da6d82e-2830-4bd6-af9e-c81db8db51e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:53.701283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da6d82e-2830-4bd6-af9e-c81db8db51e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:53.701790 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da6d82e-2830-4bd6-af9e-c81db8db51e3 admin admin] PATH_INFO: `/v3/projects/49bc5f90455e43cdafc1d2b452bd4c9f/users/700b9fc94b0a4e5f99e86fbe4410b0fe/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:53.707898 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da6d82e-2830-4bd6-af9e-c81db8db51e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49bc5f90455e43cdafc1d2b452bd4c9f, user_id=700b9fc94b0a4e5f99e86fbe4410b0fe, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:53.714193 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce99dca-7bfa-41e9-b287-32e9b477dfb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:53.714540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da6d82e-2830-4bd6-af9e-c81db8db51e3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:53.714849 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce99dca-7bfa-41e9-b287-32e9b477dfb8 service nova] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:53.715137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce99dca-7bfa-41e9-b287-32e9b477dfb8 service nova] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:53.715416 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce99dca-7bfa-41e9-b287-32e9b477dfb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:53.752540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 807/1627] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 04:59:53 2022] PUT /identity/v3/projects/49bc5f90455e43cdafc1d2b452bd4c9f/users/700b9fc94b0a4e5f99e86fbe4410b0fe/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:59:53.755294 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851337f6-8e06-4432-ab1e-069acb18039a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:53.757052 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851337f6-8e06-4432-ab1e-069acb18039a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:53.757613 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851337f6-8e06-4432-ab1e-069acb18039a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:53.821517 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce99dca-7bfa-41e9-b287-32e9b477dfb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:53.822763 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce99dca-7bfa-41e9-b287-32e9b477dfb8 service nova] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:53.824329 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-851337f6-8e06-4432-ab1e-069acb18039a None None] MFA Rules not processed for user `700b9fc94b0a4e5f99e86fbe4410b0fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:53.825639 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-851337f6-8e06-4432-ab1e-069acb18039a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:53.835632 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 821/1628] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:59:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:59:54.054349 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 808/1629] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:53 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:54.069606 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce99dca-7bfa-41e9-b287-32e9b477dfb8 service nova] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:54.119389 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:59:54.158408 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30adcb3-62d5-4d78-a6a8-c12a2162198c None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:54.158698 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30adcb3-62d5-4d78-a6a8-c12a2162198c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:54.159062 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30adcb3-62d5-4d78-a6a8-c12a2162198c None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:54.162013 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 809/1630] 10.222.0.17 () {54 vars in 912 bytes} [Tue Jul 26 04:59:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 04:59:54.182104 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cac29-1414-458b-bac4-507d767bc345 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:54.182541 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cac29-1414-458b-bac4-507d767bc345 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:54.182870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cac29-1414-458b-bac4-507d767bc345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:54.219621 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce99dca-7bfa-41e9-b287-32e9b477dfb8 service nova] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:54.225731 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-745cac29-1414-458b-bac4-507d767bc345 None None] MFA Rules not processed for user `2faeed9002ee4154bc980c146cd06bcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:54.226998 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-745cac29-1414-458b-bac4-507d767bc345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:54.256853 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:59:54.305479 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f442cbdf-da9e-41ed-9f89-d04a2c379572 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:54.307666 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442cbdf-da9e-41ed-9f89-d04a2c379572 service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:54.308704 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442cbdf-da9e-41ed-9f89-d04a2c379572 service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:54.309095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442cbdf-da9e-41ed-9f89-d04a2c379572 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:54.321693 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f442cbdf-da9e-41ed-9f89-d04a2c379572 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:54.323805 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f442cbdf-da9e-41ed-9f89-d04a2c379572 service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:54.354754 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 822/1631] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:59:54 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:59:54.403994 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 810/1632] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jul 26 04:59:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 04:59:54.417757 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745cac29-1414-458b-bac4-507d767bc345 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:54.418473 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a106ae9-2cb7-4f93-8e3e-e5c464e95482 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:54.419088 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a106ae9-2cb7-4f93-8e3e-e5c464e95482 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:54.419763 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a106ae9-2cb7-4f93-8e3e-e5c464e95482 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:54.421482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 823/1633] 10.222.0.17 () {54 vars in 912 bytes} [Tue Jul 26 04:59:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 04:59:54.431521 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745cac29-1414-458b-bac4-507d767bc345 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:54.434993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e5354-6bba-447f-b3c9-66ce82122029 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:54.435431 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e5354-6bba-447f-b3c9-66ce82122029 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:54.435846 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e5354-6bba-447f-b3c9-66ce82122029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:54.446750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9284e3-aaba-40e9-ad05-6a9b01aa1d3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:54.446750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9284e3-aaba-40e9-ad05-6a9b01aa1d3b service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:54.447253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9284e3-aaba-40e9-ad05-6a9b01aa1d3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:54.447414 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9284e3-aaba-40e9-ad05-6a9b01aa1d3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:54.480106 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9284e3-aaba-40e9-ad05-6a9b01aa1d3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:54.481616 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9284e3-aaba-40e9-ad05-6a9b01aa1d3b service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:54.506660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 811/1634] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:59:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:59:54.535143 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-129e5354-6bba-447f-b3c9-66ce82122029 None None] MFA Rules not processed for user `2faeed9002ee4154bc980c146cd06bcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:54.539458 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-129e5354-6bba-447f-b3c9-66ce82122029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:54.593637 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 824/1635] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jul 26 04:59:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 04:59:54.638504 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9284e3-aaba-40e9-ad05-6a9b01aa1d3b service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:54.663193 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9284e3-aaba-40e9-ad05-6a9b01aa1d3b service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:54.668790 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e68fbc-cdd3-4444-a9dd-142198a1a19f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:54.669671 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e68fbc-cdd3-4444-a9dd-142198a1a19f service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:54.670125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e68fbc-cdd3-4444-a9dd-142198a1a19f service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:54.670704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e68fbc-cdd3-4444-a9dd-142198a1a19f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:54.690940 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e68fbc-cdd3-4444-a9dd-142198a1a19f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:54.693390 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e68fbc-cdd3-4444-a9dd-142198a1a19f service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:54.721372 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 812/1636] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:59:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 04:59:54.966204 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e5354-6bba-447f-b3c9-66ce82122029 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:55.021625 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e68fbc-cdd3-4444-a9dd-142198a1a19f service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:55.132746 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e5354-6bba-447f-b3c9-66ce82122029 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:55.157641 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb44b00-46fd-4779-8e6c-f9b42ac173dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:55.159084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb44b00-46fd-4779-8e6c-f9b42ac173dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:55.159581 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb44b00-46fd-4779-8e6c-f9b42ac173dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:55.160124 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb44b00-46fd-4779-8e6c-f9b42ac173dc admin admin] PATH_INFO: `/v3/users/55c6c3c7738e477eb1edc784967559ee` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:55.168053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb44b00-46fd-4779-8e6c-f9b42ac173dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=55c6c3c7738e477eb1edc784967559ee)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:55.177542 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb44b00-46fd-4779-8e6c-f9b42ac173dc admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:55.191690 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e68fbc-cdd3-4444-a9dd-142198a1a19f service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:55.213607 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23c057-93b5-4e8f-bedb-d881fdaeac06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:55.214308 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23c057-93b5-4e8f-bedb-d881fdaeac06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:55.214595 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23c057-93b5-4e8f-bedb-d881fdaeac06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:55.214881 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23c057-93b5-4e8f-bedb-d881fdaeac06 admin admin] PATH_INFO: `/v3/users/92a01d413b71400da5885d4ea44ec03a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:55.216660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba23c057-93b5-4e8f-bedb-d881fdaeac06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92a01d413b71400da5885d4ea44ec03a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:55.217800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-ba23c057-93b5-4e8f-bedb-d881fdaeac06 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 04:59:55.217800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jul 26 04:59:55.217800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: Jul 26 04:59:55.219198 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba23c057-93b5-4e8f-bedb-d881fdaeac06 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:55.365458 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9cb44b00-46fd-4779-8e6c-f9b42ac173dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55c6c3c7738e477eb1edc784967559ee'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:55.381181 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ba23c057-93b5-4e8f-bedb-d881fdaeac06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92a01d413b71400da5885d4ea44ec03a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:55.394025 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9cb44b00-46fd-4779-8e6c-f9b42ac173dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55c6c3c7738e477eb1edc784967559ee'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:55.394160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ba23c057-93b5-4e8f-bedb-d881fdaeac06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92a01d413b71400da5885d4ea44ec03a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:55.394901 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ba23c057-93b5-4e8f-bedb-d881fdaeac06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92a01d413b71400da5885d4ea44ec03a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:55.419319 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9cb44b00-46fd-4779-8e6c-f9b42ac173dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55c6c3c7738e477eb1edc784967559ee'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:55.420680 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9cb44b00-46fd-4779-8e6c-f9b42ac173dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55c6c3c7738e477eb1edc784967559ee'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:55.423122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ba23c057-93b5-4e8f-bedb-d881fdaeac06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92a01d413b71400da5885d4ea44ec03a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:55.423642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9cb44b00-46fd-4779-8e6c-f9b42ac173dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55c6c3c7738e477eb1edc784967559ee'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:55.424814 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ba23c057-93b5-4e8f-bedb-d881fdaeac06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92a01d413b71400da5885d4ea44ec03a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:55.479584 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:59:55.481272 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 813/1637] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 04:59:55 2022] DELETE /identity/v3/users/92a01d413b71400da5885d4ea44ec03a => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:59:55.487611 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96049b3c-77cd-44cf-9ef0-7a0b2e6ede2f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:55.488079 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96049b3c-77cd-44cf-9ef0-7a0b2e6ede2f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:55.488431 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96049b3c-77cd-44cf-9ef0-7a0b2e6ede2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:55.499604 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 825/1638] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 04:59:54 2022] DELETE /identity/v3/users/55c6c3c7738e477eb1edc784967559ee => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:59:55.508685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d88aa-e38a-4a31-82a5-d890bed28537 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:55.508685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d88aa-e38a-4a31-82a5-d890bed28537 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:55.509042 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d88aa-e38a-4a31-82a5-d890bed28537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:55.509499 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:59:55.553043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-96049b3c-77cd-44cf-9ef0-7a0b2e6ede2f None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:55.556486 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-96049b3c-77cd-44cf-9ef0-7a0b2e6ede2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:55.559085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-a17d88aa-e38a-4a31-82a5-d890bed28537 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:55.560615 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a17d88aa-e38a-4a31-82a5-d890bed28537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:55.719188 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 826/1639] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:55.719398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 814/1640] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:55.724881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d88aa-e38a-4a31-82a5-d890bed28537 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:55.728523 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96049b3c-77cd-44cf-9ef0-7a0b2e6ede2f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:55.731478 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-96049b3c-77cd-44cf-9ef0-7a0b2e6ede2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:55.732547 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a17d88aa-e38a-4a31-82a5-d890bed28537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:55.833353 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96049b3c-77cd-44cf-9ef0-7a0b2e6ede2f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:55.842134 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2efea-9ab4-45c7-98db-47bd55c9bcff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:55.842788 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2efea-9ab4-45c7-98db-47bd55c9bcff service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:55.843083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2efea-9ab4-45c7-98db-47bd55c9bcff service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:55.843427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2efea-9ab4-45c7-98db-47bd55c9bcff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:55.849501 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d88aa-e38a-4a31-82a5-d890bed28537 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:55.855073 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2efea-9ab4-45c7-98db-47bd55c9bcff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:55.856745 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2efea-9ab4-45c7-98db-47bd55c9bcff service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:55.857998 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613393a-4605-401a-bc42-4b97787527ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:55.858919 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613393a-4605-401a-bc42-4b97787527ab service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:55.859207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613393a-4605-401a-bc42-4b97787527ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:55.859484 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613393a-4605-401a-bc42-4b97787527ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:55.869094 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 815/1641] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:59:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:59:55.886350 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8613393a-4605-401a-bc42-4b97787527ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:55.890027 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8613393a-4605-401a-bc42-4b97787527ab service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:55.906455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 827/1642] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:59:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:59:56.708411 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2efea-9ab4-45c7-98db-47bd55c9bcff service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:56.710062 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4fb2efea-9ab4-45c7-98db-47bd55c9bcff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:56.770813 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2efea-9ab4-45c7-98db-47bd55c9bcff service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:56.771648 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613393a-4605-401a-bc42-4b97787527ab service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:56.771648 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-8613393a-4605-401a-bc42-4b97787527ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:56.789341 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b255763-9a6d-472b-8b47-b3fefbeee7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:56.790043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b255763-9a6d-472b-8b47-b3fefbeee7ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:56.790333 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b255763-9a6d-472b-8b47-b3fefbeee7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:56.792678 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b255763-9a6d-472b-8b47-b3fefbeee7ce admin admin] PATH_INFO: `/v3/projects/b8af7edda5f24572bdbf92e950c73ff5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:56.794665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b255763-9a6d-472b-8b47-b3fefbeee7ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8af7edda5f24572bdbf92e950c73ff5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:56.796773 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b255763-9a6d-472b-8b47-b3fefbeee7ce admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:56.805336 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613393a-4605-401a-bc42-4b97787527ab service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:56.819816 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f71d5-c1ae-4b6a-85b2-8a4aecfcbf62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:56.821517 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f71d5-c1ae-4b6a-85b2-8a4aecfcbf62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:56.822503 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f71d5-c1ae-4b6a-85b2-8a4aecfcbf62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:56.825611 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f71d5-c1ae-4b6a-85b2-8a4aecfcbf62 admin admin] PATH_INFO: `/v3/projects/8653b1f3efea450dae2a0881cb757c86` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:56.830961 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f71d5-c1ae-4b6a-85b2-8a4aecfcbf62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8653b1f3efea450dae2a0881cb757c86)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:56.834500 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: WARNING py.warnings [None req-ce1f71d5-c1ae-4b6a-85b2-8a4aecfcbf62 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 04:59:56.834500 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: warnings.warn(msg) Jul 26 04:59:56.834500 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: Jul 26 04:59:56.842324 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f71d5-c1ae-4b6a-85b2-8a4aecfcbf62 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:56.947463 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4b255763-9a6d-472b-8b47-b3fefbeee7ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8af7edda5f24572bdbf92e950c73ff5', 'request_id': 'req-4b255763-9a6d-472b-8b47-b3fefbeee7ce'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:56.971157 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4b255763-9a6d-472b-8b47-b3fefbeee7ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8af7edda5f24572bdbf92e950c73ff5', 'request_id': 'req-4b255763-9a6d-472b-8b47-b3fefbeee7ce'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:56.999761 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4b255763-9a6d-472b-8b47-b3fefbeee7ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8af7edda5f24572bdbf92e950c73ff5', 'request_id': 'req-4b255763-9a6d-472b-8b47-b3fefbeee7ce'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:57.023278 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4b255763-9a6d-472b-8b47-b3fefbeee7ce admin admin] The token cache is being invalidate because project b8af7edda5f24572bdbf92e950c73ff5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 04:59:57.023832 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4b255763-9a6d-472b-8b47-b3fefbeee7ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:57.026446 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4b255763-9a6d-472b-8b47-b3fefbeee7ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:57.029219 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 816/1643] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 04:59:56 2022] DELETE /identity/v3/projects/b8af7edda5f24572bdbf92e950c73ff5 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:59:57.037629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b255763-9a6d-472b-8b47-b3fefbeee7ce admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:57.040121 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4b255763-9a6d-472b-8b47-b3fefbeee7ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:57.114553 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:59:57.137034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-ce1f71d5-c1ae-4b6a-85b2-8a4aecfcbf62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8653b1f3efea450dae2a0881cb757c86', 'request_id': 'req-ce1f71d5-c1ae-4b6a-85b2-8a4aecfcbf62'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:57.156662 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-ce1f71d5-c1ae-4b6a-85b2-8a4aecfcbf62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8653b1f3efea450dae2a0881cb757c86', 'request_id': 'req-ce1f71d5-c1ae-4b6a-85b2-8a4aecfcbf62'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:57.196640 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b255763-9a6d-472b-8b47-b3fefbeee7ce admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:57.211055 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-ce1f71d5-c1ae-4b6a-85b2-8a4aecfcbf62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8653b1f3efea450dae2a0881cb757c86', 'request_id': 'req-ce1f71d5-c1ae-4b6a-85b2-8a4aecfcbf62'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:57.253740 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-ce1f71d5-c1ae-4b6a-85b2-8a4aecfcbf62 admin admin] The token cache is being invalidate because project 8653b1f3efea450dae2a0881cb757c86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 04:59:57.253971 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-ce1f71d5-c1ae-4b6a-85b2-8a4aecfcbf62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:57.256425 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-ce1f71d5-c1ae-4b6a-85b2-8a4aecfcbf62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:57.263428 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 828/1644] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 04:59:56 2022] DELETE /identity/v3/projects/8653b1f3efea450dae2a0881cb757c86 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:59:57.265094 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51372a33-a651-4227-bb53-1bbe0276cd9b None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:57.265369 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51372a33-a651-4227-bb53-1bbe0276cd9b None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:57.265638 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51372a33-a651-4227-bb53-1bbe0276cd9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:57.330589 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-51372a33-a651-4227-bb53-1bbe0276cd9b None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:57.331898 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-51372a33-a651-4227-bb53-1bbe0276cd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:57.381907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:59:57.395272 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79abc4-4246-4a79-a531-f2bdd494a095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:57.396287 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79abc4-4246-4a79-a531-f2bdd494a095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:57.396716 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79abc4-4246-4a79-a531-f2bdd494a095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:57.397125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79abc4-4246-4a79-a531-f2bdd494a095 admin admin] PATH_INFO: `/v3/users/700b9fc94b0a4e5f99e86fbe4410b0fe` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:57.399160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79abc4-4246-4a79-a531-f2bdd494a095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=700b9fc94b0a4e5f99e86fbe4410b0fe)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:57.401474 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79abc4-4246-4a79-a531-f2bdd494a095 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:57.561011 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0d79abc4-4246-4a79-a531-f2bdd494a095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '700b9fc94b0a4e5f99e86fbe4410b0fe'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:57.579227 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 829/1645] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:57.584598 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0d79abc4-4246-4a79-a531-f2bdd494a095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '700b9fc94b0a4e5f99e86fbe4410b0fe'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:57.589790 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0d79abc4-4246-4a79-a531-f2bdd494a095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '700b9fc94b0a4e5f99e86fbe4410b0fe'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:57.591331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebf59a-b19e-4582-8251-005d2bc86451 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:57.591417 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebf59a-b19e-4582-8251-005d2bc86451 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:57.591739 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebf59a-b19e-4582-8251-005d2bc86451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:57.607450 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0d79abc4-4246-4a79-a531-f2bdd494a095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '700b9fc94b0a4e5f99e86fbe4410b0fe'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:57.608448 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0d79abc4-4246-4a79-a531-f2bdd494a095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '700b9fc94b0a4e5f99e86fbe4410b0fe'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:57.634384 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-c4ebf59a-b19e-4582-8251-005d2bc86451 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:57.636489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c4ebf59a-b19e-4582-8251-005d2bc86451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:57.652724 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 830/1646] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:57.658795 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ebf59a-b19e-4582-8251-005d2bc86451 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:57.665251 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 817/1647] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 04:59:57 2022] DELETE /identity/v3/users/700b9fc94b0a4e5f99e86fbe4410b0fe => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:59:57.671968 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ebf59a-b19e-4582-8251-005d2bc86451 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:57.757550 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea46dc5-022e-4049-9cb3-7cf8375d7a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:57.758941 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea46dc5-022e-4049-9cb3-7cf8375d7a88 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:57.759345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea46dc5-022e-4049-9cb3-7cf8375d7a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:57.759832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea46dc5-022e-4049-9cb3-7cf8375d7a88 admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:57.761094 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea46dc5-022e-4049-9cb3-7cf8375d7a88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:59:57.761740 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea46dc5-022e-4049-9cb3-7cf8375d7a88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:57.764757 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea46dc5-022e-4049-9cb3-7cf8375d7a88 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:57.792253 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 831/1648] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 04:59:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 04:59:57.800321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79abc4-4246-4a79-a531-f2bdd494a095 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:57.826709 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79abc4-4246-4a79-a531-f2bdd494a095 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:57.838190 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb399af-a033-42c5-89f4-949af072dd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:57.840019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb399af-a033-42c5-89f4-949af072dd2a admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:57.840993 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb399af-a033-42c5-89f4-949af072dd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:57.841596 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb399af-a033-42c5-89f4-949af072dd2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:57.843075 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb399af-a033-42c5-89f4-949af072dd2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:57.847431 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb399af-a033-42c5-89f4-949af072dd2a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:57.917676 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 818/1649] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 04:59:57 2022] POST /identity/v3/projects => generated 405 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 04:59:57.918778 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea46dc5-022e-4049-9cb3-7cf8375d7a88 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:57.955007 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea46dc5-022e-4049-9cb3-7cf8375d7a88 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:58.094300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997c76c-bcc4-4bfc-b4c5-3e580297de9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:58.095483 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997c76c-bcc4-4bfc-b4c5-3e580297de9e admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:58.096010 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997c76c-bcc4-4bfc-b4c5-3e580297de9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:58.096542 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997c76c-bcc4-4bfc-b4c5-3e580297de9e admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:58.097402 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6997c76c-bcc4-4bfc-b4c5-3e580297de9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:58.100479 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6997c76c-bcc4-4bfc-b4c5-3e580297de9e admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:58.163920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 832/1650] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 04:59:57 2022] POST /identity/v3/users => generated 303 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:59:58.170726 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb399af-a033-42c5-89f4-949af072dd2a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:58.190617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb399af-a033-42c5-89f4-949af072dd2a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:58.200171 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c9343-0ffc-49e4-87e9-fd85975a6a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:58.201631 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c9343-0ffc-49e4-87e9-fd85975a6a52 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:58.202310 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c9343-0ffc-49e4-87e9-fd85975a6a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:58.203351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c9343-0ffc-49e4-87e9-fd85975a6a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:58.211840 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5c9343-0ffc-49e4-87e9-fd85975a6a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:58.215336 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5c9343-0ffc-49e4-87e9-fd85975a6a52 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:58.246749 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 819/1651] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 04:59:58 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 04:59:58.247735 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997c76c-bcc4-4bfc-b4c5-3e580297de9e admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:58.264369 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997c76c-bcc4-4bfc-b4c5-3e580297de9e admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:58.271321 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707224a2-0b00-448a-af99-fd97ce8267b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:58.271861 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707224a2-0b00-448a-af99-fd97ce8267b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:58.272253 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707224a2-0b00-448a-af99-fd97ce8267b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:58.272626 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707224a2-0b00-448a-af99-fd97ce8267b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:58.274625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707224a2-0b00-448a-af99-fd97ce8267b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:58.276564 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707224a2-0b00-448a-af99-fd97ce8267b8 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:58.291220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 833/1652] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 04:59:58 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 04:59:58.297935 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c9343-0ffc-49e4-87e9-fd85975a6a52 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:58.328978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c9343-0ffc-49e4-87e9-fd85975a6a52 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:58.350813 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9ebad-a178-460d-a773-541d5389b492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:58.352765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9ebad-a178-460d-a773-541d5389b492 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:58.359057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9ebad-a178-460d-a773-541d5389b492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:58.359776 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9ebad-a178-460d-a773-541d5389b492 admin admin] PATH_INFO: `/v3/projects/4e1d08f366f24923bb7b43a52178d51e/users/d9e53fc3e5e345f0b9bc6efa3745441f/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:58.401629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9ebad-a178-460d-a773-541d5389b492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e1d08f366f24923bb7b43a52178d51e, user_id=d9e53fc3e5e345f0b9bc6efa3745441f, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:58.414467 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9ebad-a178-460d-a773-541d5389b492 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:58.429436 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:59:58.442899 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707224a2-0b00-448a-af99-fd97ce8267b8 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:58.445073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-707224a2-0b00-448a-af99-fd97ce8267b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:58.450315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 820/1653] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 04:59:58 2022] PUT /identity/v3/projects/4e1d08f366f24923bb7b43a52178d51e/users/d9e53fc3e5e345f0b9bc6efa3745441f/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:59:58.454545 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ca066-0304-4254-8266-20d9dc04dd97 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:58.455525 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ca066-0304-4254-8266-20d9dc04dd97 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:58.456506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ca066-0304-4254-8266-20d9dc04dd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:58.471732 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707224a2-0b00-448a-af99-fd97ce8267b8 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:58.530159 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f59ca066-0304-4254-8266-20d9dc04dd97 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:58.533311 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f59ca066-0304-4254-8266-20d9dc04dd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:58.575722 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:59:58.769068 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432cb685-a274-41f6-aa9e-13ad2555a093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:58.780074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432cb685-a274-41f6-aa9e-13ad2555a093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:58.780633 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432cb685-a274-41f6-aa9e-13ad2555a093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:58.782207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432cb685-a274-41f6-aa9e-13ad2555a093 admin admin] PATH_INFO: `/v3/users/c84bbfe30cb04dd5adcd67987e31412d` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:58.788981 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432cb685-a274-41f6-aa9e-13ad2555a093 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c84bbfe30cb04dd5adcd67987e31412d)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:58.805782 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432cb685-a274-41f6-aa9e-13ad2555a093 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:58.818570 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 821/1654] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:58.906233 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ca066-0304-4254-8266-20d9dc04dd97 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:58.908596 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f59ca066-0304-4254-8266-20d9dc04dd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:58.964730 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-432cb685-a274-41f6-aa9e-13ad2555a093 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c84bbfe30cb04dd5adcd67987e31412d'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:58.990051 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-432cb685-a274-41f6-aa9e-13ad2555a093 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c84bbfe30cb04dd5adcd67987e31412d'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:59.015677 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-432cb685-a274-41f6-aa9e-13ad2555a093 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c84bbfe30cb04dd5adcd67987e31412d'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:59.018888 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-432cb685-a274-41f6-aa9e-13ad2555a093 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c84bbfe30cb04dd5adcd67987e31412d'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:59.020649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-432cb685-a274-41f6-aa9e-13ad2555a093 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c84bbfe30cb04dd5adcd67987e31412d'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:59.050295 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ca066-0304-4254-8266-20d9dc04dd97 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:59.061739 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7cd06f-2cac-4aa4-9630-9161d751810d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:59.062767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7cd06f-2cac-4aa4-9630-9161d751810d service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:59.063262 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7cd06f-2cac-4aa4-9630-9161d751810d service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:59.063732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7cd06f-2cac-4aa4-9630-9161d751810d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:59.071801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ac7cd06f-2cac-4aa4-9630-9161d751810d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:59.099047 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7cd06f-2cac-4aa4-9630-9161d751810d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:59.100675 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 834/1655] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 04:59:58 2022] DELETE /identity/v3/users/c84bbfe30cb04dd5adcd67987e31412d => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:59:59.102244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1ed9e-b00b-478f-baa9-f6af088a500e None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:59.102582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1ed9e-b00b-478f-baa9-f6af088a500e None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:59.103027 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1ed9e-b00b-478f-baa9-f6af088a500e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:59.107105 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7cd06f-2cac-4aa4-9630-9161d751810d service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:59.129680 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 822/1656] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:59:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 04:59:59.133581 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7cd06f-2cac-4aa4-9630-9161d751810d service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:59.133934 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-c4e1ed9e-b00b-478f-baa9-f6af088a500e None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:59.135650 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c4e1ed9e-b00b-478f-baa9-f6af088a500e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:59.136295 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ac7cd06f-2cac-4aa4-9630-9161d751810d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:59.162431 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7cd06f-2cac-4aa4-9630-9161d751810d service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:59.316963 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 835/1657] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:59.324530 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79d801-ef56-49ff-ba04-a8f7ac98b011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:59.325583 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79d801-ef56-49ff-ba04-a8f7ac98b011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:59.332290 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79d801-ef56-49ff-ba04-a8f7ac98b011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:59.332937 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79d801-ef56-49ff-ba04-a8f7ac98b011 admin admin] PATH_INFO: `/v3/projects/49bc5f90455e43cdafc1d2b452bd4c9f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:59.334924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd79d801-ef56-49ff-ba04-a8f7ac98b011 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49bc5f90455e43cdafc1d2b452bd4c9f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:59.337233 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd79d801-ef56-49ff-ba04-a8f7ac98b011 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:59.361496 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1ed9e-b00b-478f-baa9-f6af088a500e None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:59.363455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c4e1ed9e-b00b-478f-baa9-f6af088a500e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:59.446337 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:59:59.455608 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dd79d801-ef56-49ff-ba04-a8f7ac98b011 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49bc5f90455e43cdafc1d2b452bd4c9f', 'request_id': 'req-dd79d801-ef56-49ff-ba04-a8f7ac98b011'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:59.471962 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dd79d801-ef56-49ff-ba04-a8f7ac98b011 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49bc5f90455e43cdafc1d2b452bd4c9f', 'request_id': 'req-dd79d801-ef56-49ff-ba04-a8f7ac98b011'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:59.491842 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dd79d801-ef56-49ff-ba04-a8f7ac98b011 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49bc5f90455e43cdafc1d2b452bd4c9f', 'request_id': 'req-dd79d801-ef56-49ff-ba04-a8f7ac98b011'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:59.517583 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dd79d801-ef56-49ff-ba04-a8f7ac98b011 admin admin] The token cache is being invalidate because project 49bc5f90455e43cdafc1d2b452bd4c9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 04:59:59.517888 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dd79d801-ef56-49ff-ba04-a8f7ac98b011 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:59.519089 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dd79d801-ef56-49ff-ba04-a8f7ac98b011 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 04:59:59.524417 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 823/1658] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 04:59:59 2022] DELETE /identity/v3/projects/49bc5f90455e43cdafc1d2b452bd4c9f => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 04:59:59.528927 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1ed9e-b00b-478f-baa9-f6af088a500e None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:59.529223 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2caefbb-71c1-4f20-a868-b212398b4a42 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:59.529683 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2caefbb-71c1-4f20-a868-b212398b4a42 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:59.530060 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2caefbb-71c1-4f20-a868-b212398b4a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:59.535314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 04:59:59.588298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c2caefbb-71c1-4f20-a868-b212398b4a42 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:59.590924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c2caefbb-71c1-4f20-a868-b212398b4a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:59.675850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52865f8c-3d58-496a-b68f-c33b7953d86a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:59.678265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52865f8c-3d58-496a-b68f-c33b7953d86a service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:59.678557 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52865f8c-3d58-496a-b68f-c33b7953d86a service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:59.678804 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52865f8c-3d58-496a-b68f-c33b7953d86a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:59.683237 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-52865f8c-3d58-496a-b68f-c33b7953d86a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:59.684295 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 824/1659] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:59.694532 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d47b38-815f-4e20-8270-a6410f256760 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:59.694961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d47b38-815f-4e20-8270-a6410f256760 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:59.695098 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d47b38-815f-4e20-8270-a6410f256760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:59.724016 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52865f8c-3d58-496a-b68f-c33b7953d86a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:59.725614 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52865f8c-3d58-496a-b68f-c33b7953d86a service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:59.732134 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-09d47b38-815f-4e20-8270-a6410f256760 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 04:59:59.733799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-09d47b38-815f-4e20-8270-a6410f256760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 04:59:59.751029 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 836/1660] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 04:59:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 04:59:59.766439 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 825/1661] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 04:59:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 04:59:59.768039 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52865f8c-3d58-496a-b68f-c33b7953d86a service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:59.781805 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52865f8c-3d58-496a-b68f-c33b7953d86a service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:59.793472 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ae7d2-b68a-4b19-af30-25e0ebf63c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:59.794452 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ae7d2-b68a-4b19-af30-25e0ebf63c1d admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:59.795046 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ae7d2-b68a-4b19-af30-25e0ebf63c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:59.796344 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ae7d2-b68a-4b19-af30-25e0ebf63c1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:59.797854 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ae7d2-b68a-4b19-af30-25e0ebf63c1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 04:59:59.798952 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ae7d2-b68a-4b19-af30-25e0ebf63c1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:59.804203 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ae7d2-b68a-4b19-af30-25e0ebf63c1d admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:59.834160 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 837/1662] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 04:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 04:59:59.836285 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d47b38-815f-4e20-8270-a6410f256760 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:59.851383 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d47b38-815f-4e20-8270-a6410f256760 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 04:59:59.872858 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb590b-1e87-4a4e-bb29-3c34936de24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 04:59:59.873531 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb590b-1e87-4a4e-bb29-3c34936de24e admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 04:59:59.873815 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb590b-1e87-4a4e-bb29-3c34936de24e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 04:59:59.874090 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb590b-1e87-4a4e-bb29-3c34936de24e admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 04:59:59.874801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bb590b-1e87-4a4e-bb29-3c34936de24e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 04:59:59.876825 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bb590b-1e87-4a4e-bb29-3c34936de24e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 04:59:59.958372 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 826/1663] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 04:59:59 2022] POST /identity/v3/projects => generated 365 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 04:59:59.959954 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ae7d2-b68a-4b19-af30-25e0ebf63c1d admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 04:59:59.990438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ae7d2-b68a-4b19-af30-25e0ebf63c1d admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:00.222270 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb6870-4d83-4f2f-9b0f-d5d16fbd4197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:00.225120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb6870-4d83-4f2f-9b0f-d5d16fbd4197 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:00.225900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb6870-4d83-4f2f-9b0f-d5d16fbd4197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:00.226788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb6870-4d83-4f2f-9b0f-d5d16fbd4197 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:00.229201 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb6870-4d83-4f2f-9b0f-d5d16fbd4197 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:00.235421 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb6870-4d83-4f2f-9b0f-d5d16fbd4197 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:00.303305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb590b-1e87-4a4e-bb29-3c34936de24e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:00.319708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 838/1664] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 04:59:59 2022] POST /identity/v3/users => generated 283 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:00.338821 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb590b-1e87-4a4e-bb29-3c34936de24e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:00.356361 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582b1f4-58fd-41cc-8a08-be2272a93fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:00.358070 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582b1f4-58fd-41cc-8a08-be2272a93fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:00.359206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582b1f4-58fd-41cc-8a08-be2272a93fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:00.359821 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582b1f4-58fd-41cc-8a08-be2272a93fe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:00.361215 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1582b1f4-58fd-41cc-8a08-be2272a93fe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:00.368988 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1582b1f4-58fd-41cc-8a08-be2272a93fe1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:00.412300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb6870-4d83-4f2f-9b0f-d5d16fbd4197 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:00.421794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 827/1665] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:00 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:00.440026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb6870-4d83-4f2f-9b0f-d5d16fbd4197 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:00.449752 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06eefa1-c139-42a9-bee9-e3805866ad47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:00.451416 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06eefa1-c139-42a9-bee9-e3805866ad47 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:00.451988 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06eefa1-c139-42a9-bee9-e3805866ad47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:00.452428 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06eefa1-c139-42a9-bee9-e3805866ad47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:00.453746 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06eefa1-c139-42a9-bee9-e3805866ad47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:00.456304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06eefa1-c139-42a9-bee9-e3805866ad47 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:00.488233 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582b1f4-58fd-41cc-8a08-be2272a93fe1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:00.500344 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 839/1666] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:00 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:00.508188 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582b1f4-58fd-41cc-8a08-be2272a93fe1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:00.524098 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba6b93-7ea1-4c34-8330-3fdbe3f46bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:00.526620 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba6b93-7ea1-4c34-8330-3fdbe3f46bfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:00.527284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba6b93-7ea1-4c34-8330-3fdbe3f46bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:00.527990 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba6b93-7ea1-4c34-8330-3fdbe3f46bfe admin admin] PATH_INFO: `/v3/projects/88f35a8508a2419d84734748589ea81a/users/154ade73bc7a40458078170cc73d0634/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:00.580810 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba6b93-7ea1-4c34-8330-3fdbe3f46bfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=88f35a8508a2419d84734748589ea81a, user_id=154ade73bc7a40458078170cc73d0634, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:00.584500 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba6b93-7ea1-4c34-8330-3fdbe3f46bfe admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:00.645827 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 828/1667] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:00 2022] PUT /identity/v3/projects/88f35a8508a2419d84734748589ea81a/users/154ade73bc7a40458078170cc73d0634/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:00.654242 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48065881-e0cc-4297-bda2-7d3352765165 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:00.654814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48065881-e0cc-4297-bda2-7d3352765165 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:00.655289 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48065881-e0cc-4297-bda2-7d3352765165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:00.720871 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-48065881-e0cc-4297-bda2-7d3352765165 None None] MFA Rules not processed for user `154ade73bc7a40458078170cc73d0634`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:00.725123 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-48065881-e0cc-4297-bda2-7d3352765165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:00.890475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 840/1668] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:00 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:00.892321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba6b93-7ea1-4c34-8330-3fdbe3f46bfe admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:01.156861 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba6b93-7ea1-4c34-8330-3fdbe3f46bfe admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:01.179442 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b0878-39f0-4b36-b618-1243473b0287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:01.181088 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b0878-39f0-4b36-b618-1243473b0287 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:01.181425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b0878-39f0-4b36-b618-1243473b0287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:01.182384 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b0878-39f0-4b36-b618-1243473b0287 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:01.183142 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b0878-39f0-4b36-b618-1243473b0287 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:01.192588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b0878-39f0-4b36-b618-1243473b0287 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:01.246255 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:01.280158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 829/1669] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:00:00 2022] POST /identity/v3/projects => generated 363 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:01.286393 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48065881-e0cc-4297-bda2-7d3352765165 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:01.296410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:01.306047 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48065881-e0cc-4297-bda2-7d3352765165 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:01.548840 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebc5fe-ef53-41aa-b01e-30db0eb488fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:01.549584 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebc5fe-ef53-41aa-b01e-30db0eb488fa admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:01.549879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebc5fe-ef53-41aa-b01e-30db0eb488fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:01.550160 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebc5fe-ef53-41aa-b01e-30db0eb488fa admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:01.550880 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebc5fe-ef53-41aa-b01e-30db0eb488fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:01.553110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebc5fe-ef53-41aa-b01e-30db0eb488fa admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:01.639284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b0878-39f0-4b36-b618-1243473b0287 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:01.644600 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 841/1670] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:00:01 2022] POST /identity/v3/users => generated 288 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:01.661721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b0878-39f0-4b36-b618-1243473b0287 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:01.693884 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdc9ef-83fb-4bc3-9ddf-1732bc9f17de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:01.697331 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdc9ef-83fb-4bc3-9ddf-1732bc9f17de admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:01.697676 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdc9ef-83fb-4bc3-9ddf-1732bc9f17de admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:01.698106 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdc9ef-83fb-4bc3-9ddf-1732bc9f17de admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:01.701088 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bdc9ef-83fb-4bc3-9ddf-1732bc9f17de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:01.711745 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bdc9ef-83fb-4bc3-9ddf-1732bc9f17de admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:01.797228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebc5fe-ef53-41aa-b01e-30db0eb488fa admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:01.802395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 830/1671] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:01.828311 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebc5fe-ef53-41aa-b01e-30db0eb488fa admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:01.854675 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d028796-ec3d-4aaa-bbdd-81467561e762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:01.866650 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d028796-ec3d-4aaa-bbdd-81467561e762 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:01.867054 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d028796-ec3d-4aaa-bbdd-81467561e762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:01.867337 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d028796-ec3d-4aaa-bbdd-81467561e762 admin admin] PATH_INFO: `/v3/projects/7ac0576b99a24d5997072f864a86c8c0/users/5cec519e633e4a2ba70e0bac981cee6e/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:01.890246 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d028796-ec3d-4aaa-bbdd-81467561e762 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ac0576b99a24d5997072f864a86c8c0, user_id=5cec519e633e4a2ba70e0bac981cee6e, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:01.895053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d028796-ec3d-4aaa-bbdd-81467561e762 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:01.929258 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 842/1672] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:01 2022] PUT /identity/v3/projects/7ac0576b99a24d5997072f864a86c8c0/users/5cec519e633e4a2ba70e0bac981cee6e/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:01.929898 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdc9ef-83fb-4bc3-9ddf-1732bc9f17de admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:01.949838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdc9ef-83fb-4bc3-9ddf-1732bc9f17de admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:02.129705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc020571-206f-4ef3-9f12-25cbee8891b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:02.130450 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc020571-206f-4ef3-9f12-25cbee8891b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:02.130765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc020571-206f-4ef3-9f12-25cbee8891b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:02.131042 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc020571-206f-4ef3-9f12-25cbee8891b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:02.131834 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc020571-206f-4ef3-9f12-25cbee8891b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:02.133496 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc020571-206f-4ef3-9f12-25cbee8891b9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:02.154685 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 831/1673] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:00:02.160199 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d028796-ec3d-4aaa-bbdd-81467561e762 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:02.180203 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d028796-ec3d-4aaa-bbdd-81467561e762 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:02.208298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60aba58-7bc5-4906-9854-be054ab4a3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:02.209225 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60aba58-7bc5-4906-9854-be054ab4a3b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:02.209984 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60aba58-7bc5-4906-9854-be054ab4a3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:02.210371 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60aba58-7bc5-4906-9854-be054ab4a3b9 admin admin] PATH_INFO: `/v3/projects/7ac0576b99a24d5997072f864a86c8c0/users/5cec519e633e4a2ba70e0bac981cee6e/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:02.213798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60aba58-7bc5-4906-9854-be054ab4a3b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ac0576b99a24d5997072f864a86c8c0, user_id=5cec519e633e4a2ba70e0bac981cee6e, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:02.220866 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60aba58-7bc5-4906-9854-be054ab4a3b9 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:02.270290 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 843/1674] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:02 2022] PUT /identity/v3/projects/7ac0576b99a24d5997072f864a86c8c0/users/5cec519e633e4a2ba70e0bac981cee6e/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:02.274432 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b963d41-3f4a-4112-b28f-7a5867400a7a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:02.274734 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b963d41-3f4a-4112-b28f-7a5867400a7a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:02.275025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b963d41-3f4a-4112-b28f-7a5867400a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:02.366357 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8b963d41-3f4a-4112-b28f-7a5867400a7a None None] MFA Rules not processed for user `5cec519e633e4a2ba70e0bac981cee6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:02.368107 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8b963d41-3f4a-4112-b28f-7a5867400a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:02.517495 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 832/1675] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:02 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:02.531466 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60aba58-7bc5-4906-9854-be054ab4a3b9 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:02.541061 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c60aba58-7bc5-4906-9854-be054ab4a3b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:02.578256 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b963d41-3f4a-4112-b28f-7a5867400a7a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:02.579947 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8b963d41-3f4a-4112-b28f-7a5867400a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:02.776583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60aba58-7bc5-4906-9854-be054ab4a3b9 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:02.788251 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b772f-4e43-466e-b0d6-db16030db008 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:02.788918 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b772f-4e43-466e-b0d6-db16030db008 service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:02.789200 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b772f-4e43-466e-b0d6-db16030db008 service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:02.789508 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b772f-4e43-466e-b0d6-db16030db008 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:02.839443 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035b772f-4e43-466e-b0d6-db16030db008 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:02.841423 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035b772f-4e43-466e-b0d6-db16030db008 service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:02.853254 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b963d41-3f4a-4112-b28f-7a5867400a7a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:02.864165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 844/1676] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:02 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:02.869522 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ab2b9-04c4-4c14-8e1b-14b64704b7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:02.870011 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ab2b9-04c4-4c14-8e1b-14b64704b7e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:02.870379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ab2b9-04c4-4c14-8e1b-14b64704b7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:02.871845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ab2b9-04c4-4c14-8e1b-14b64704b7e1 admin admin] PATH_INFO: `/v3/projects/f797e5a15e8b4bc887794be8dc622961` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:02.875711 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7ab2b9-04c4-4c14-8e1b-14b64704b7e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f797e5a15e8b4bc887794be8dc622961)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:02.879444 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7ab2b9-04c4-4c14-8e1b-14b64704b7e1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:02.905693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:03.101876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-eb7ab2b9-04c4-4c14-8e1b-14b64704b7e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f797e5a15e8b4bc887794be8dc622961', 'request_id': 'req-eb7ab2b9-04c4-4c14-8e1b-14b64704b7e1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:03.112808 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-eb7ab2b9-04c4-4c14-8e1b-14b64704b7e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f797e5a15e8b4bc887794be8dc622961', 'request_id': 'req-eb7ab2b9-04c4-4c14-8e1b-14b64704b7e1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:03.131903 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-eb7ab2b9-04c4-4c14-8e1b-14b64704b7e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f797e5a15e8b4bc887794be8dc622961', 'request_id': 'req-eb7ab2b9-04c4-4c14-8e1b-14b64704b7e1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:03.165526 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-eb7ab2b9-04c4-4c14-8e1b-14b64704b7e1 admin admin] The token cache is being invalidate because project f797e5a15e8b4bc887794be8dc622961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:00:03.165526 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-eb7ab2b9-04c4-4c14-8e1b-14b64704b7e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:03.166677 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-eb7ab2b9-04c4-4c14-8e1b-14b64704b7e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:03.171452 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 833/1677] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:00:02 2022] DELETE /identity/v3/projects/f797e5a15e8b4bc887794be8dc622961 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:03.177866 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1fc1ff-1cc4-4965-9fd9-8b8d2ec7881c None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:03.178872 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1fc1ff-1cc4-4965-9fd9-8b8d2ec7881c None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:03.179776 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1fc1ff-1cc4-4965-9fd9-8b8d2ec7881c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:03.193726 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ab2b9-04c4-4c14-8e1b-14b64704b7e1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:03.196281 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-eb7ab2b9-04c4-4c14-8e1b-14b64704b7e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:03.218195 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-ca1fc1ff-1cc4-4965-9fd9-8b8d2ec7881c None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:03.226193 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-ca1fc1ff-1cc4-4965-9fd9-8b8d2ec7881c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:03.306725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ab2b9-04c4-4c14-8e1b-14b64704b7e1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:03.312255 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1c186-8eb4-4dcb-96c1-33352dd00362 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:03.313581 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1c186-8eb4-4dcb-96c1-33352dd00362 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:03.314351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1c186-8eb4-4dcb-96c1-33352dd00362 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:03.315128 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1c186-8eb4-4dcb-96c1-33352dd00362 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:03.318650 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-39d1c186-8eb4-4dcb-96c1-33352dd00362 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:03.324756 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:03.379822 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 845/1678] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:03.388600 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6fe4a-ed7b-407c-a4e9-68344c188e93 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:03.388600 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6fe4a-ed7b-407c-a4e9-68344c188e93 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:03.388600 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6fe4a-ed7b-407c-a4e9-68344c188e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:03.413802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-16f6fe4a-ed7b-407c-a4e9-68344c188e93 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:03.416120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-16f6fe4a-ed7b-407c-a4e9-68344c188e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:03.440248 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 846/1679] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:03.441384 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6fe4a-ed7b-407c-a4e9-68344c188e93 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:03.477003 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6fe4a-ed7b-407c-a4e9-68344c188e93 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:03.493345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34bb189-38a3-4e5d-b675-cc22b7faea69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:03.494104 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bb189-38a3-4e5d-b675-cc22b7faea69 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:03.494662 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bb189-38a3-4e5d-b675-cc22b7faea69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:03.495009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bb189-38a3-4e5d-b675-cc22b7faea69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:03.497873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34bb189-38a3-4e5d-b675-cc22b7faea69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:00:03.500486 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34bb189-38a3-4e5d-b675-cc22b7faea69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:03.501672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1c186-8eb4-4dcb-96c1-33352dd00362 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:03.504455 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1c186-8eb4-4dcb-96c1-33352dd00362 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:03.505334 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34bb189-38a3-4e5d-b675-cc22b7faea69 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:03.529459 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 834/1680] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:03 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:03.532180 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 847/1681] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:00:03.537980 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1c186-8eb4-4dcb-96c1-33352dd00362 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:03.585343 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1c186-8eb4-4dcb-96c1-33352dd00362 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:03.627317 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538674a-cee5-4042-801c-abe1db3a17cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:03.627317 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538674a-cee5-4042-801c-abe1db3a17cf admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:03.627640 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538674a-cee5-4042-801c-abe1db3a17cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:03.627857 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538674a-cee5-4042-801c-abe1db3a17cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:03.628565 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538674a-cee5-4042-801c-abe1db3a17cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:03.631093 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538674a-cee5-4042-801c-abe1db3a17cf admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:03.713987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 835/1682] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:00:03 2022] POST /identity/v3/projects => generated 365 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:03.718205 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34bb189-38a3-4e5d-b675-cc22b7faea69 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:03.744770 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34bb189-38a3-4e5d-b675-cc22b7faea69 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:03.946876 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052dc5b-59b5-4bae-89ab-01d7c354feaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:03.947102 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052dc5b-59b5-4bae-89ab-01d7c354feaa admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:03.947286 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052dc5b-59b5-4bae-89ab-01d7c354feaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:03.947838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052dc5b-59b5-4bae-89ab-01d7c354feaa admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:03.948680 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6052dc5b-59b5-4bae-89ab-01d7c354feaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:03.951019 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6052dc5b-59b5-4bae-89ab-01d7c354feaa admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:03.994205 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 848/1683] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:00:03 2022] POST /identity/v3/users => generated 283 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:04.000223 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538674a-cee5-4042-801c-abe1db3a17cf admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:04.021796 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538674a-cee5-4042-801c-abe1db3a17cf admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:04.041679 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1d78c-31de-48ed-b236-d5a4e1cd2009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:04.042443 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1d78c-31de-48ed-b236-d5a4e1cd2009 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:04.042943 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1d78c-31de-48ed-b236-d5a4e1cd2009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:04.044121 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1d78c-31de-48ed-b236-d5a4e1cd2009 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:04.045089 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1d78c-31de-48ed-b236-d5a4e1cd2009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:04.046989 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1d78c-31de-48ed-b236-d5a4e1cd2009 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:04.073967 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 836/1684] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:04.082099 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052dc5b-59b5-4bae-89ab-01d7c354feaa admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:04.100874 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052dc5b-59b5-4bae-89ab-01d7c354feaa admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:04.118532 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf4ec3-ce68-44d3-a027-f6a8c0295776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:04.119430 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf4ec3-ce68-44d3-a027-f6a8c0295776 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:04.119742 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf4ec3-ce68-44d3-a027-f6a8c0295776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:04.120019 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf4ec3-ce68-44d3-a027-f6a8c0295776 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:04.120748 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf4ec3-ce68-44d3-a027-f6a8c0295776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:04.130615 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf4ec3-ce68-44d3-a027-f6a8c0295776 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:04.203887 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 849/1685] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:04.208271 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1d78c-31de-48ed-b236-d5a4e1cd2009 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:04.225947 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbcd895-d484-4fe5-8de7-dbd97cc77081 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:04.226914 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbcd895-d484-4fe5-8de7-dbd97cc77081 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:04.227697 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbcd895-d484-4fe5-8de7-dbd97cc77081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:04.255938 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1d78c-31de-48ed-b236-d5a4e1cd2009 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:04.288854 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2712a0c-f075-4dd3-9dda-62ebbc92a442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:04.289786 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2712a0c-f075-4dd3-9dda-62ebbc92a442 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:04.290486 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2712a0c-f075-4dd3-9dda-62ebbc92a442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:04.293076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2712a0c-f075-4dd3-9dda-62ebbc92a442 admin admin] PATH_INFO: `/v3/projects/35463c490516487c936b93bcd5407ba6/users/13be1d70d27b4be3927b6ee4ee213c4f/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:04.346496 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-7dbcd895-d484-4fe5-8de7-dbd97cc77081 None None] MFA Rules not processed for user `d9e53fc3e5e345f0b9bc6efa3745441f`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:04.347547 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2712a0c-f075-4dd3-9dda-62ebbc92a442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35463c490516487c936b93bcd5407ba6, user_id=13be1d70d27b4be3927b6ee4ee213c4f, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:04.349132 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-7dbcd895-d484-4fe5-8de7-dbd97cc77081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:04.352293 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2712a0c-f075-4dd3-9dda-62ebbc92a442 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:04.367005 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:04.420452 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 837/1686] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:04 2022] PUT /identity/v3/projects/35463c490516487c936b93bcd5407ba6/users/13be1d70d27b4be3927b6ee4ee213c4f/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:04.432918 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2e77d-54c9-4166-9e4d-07b0fbe2bf15 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:04.433538 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2e77d-54c9-4166-9e4d-07b0fbe2bf15 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:04.434008 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2e77d-54c9-4166-9e4d-07b0fbe2bf15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:04.527969 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-07e2e77d-54c9-4166-9e4d-07b0fbe2bf15 None None] MFA Rules not processed for user `13be1d70d27b4be3927b6ee4ee213c4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:04.530473 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-07e2e77d-54c9-4166-9e4d-07b0fbe2bf15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:04.625800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 850/1687] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:04 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:00:04.631707 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbcd895-d484-4fe5-8de7-dbd97cc77081 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:04.645904 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-7dbcd895-d484-4fe5-8de7-dbd97cc77081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:04.698753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbcd895-d484-4fe5-8de7-dbd97cc77081 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:04.781054 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 838/1688] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:04 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:00:04.782076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e2e77d-54c9-4166-9e4d-07b0fbe2bf15 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:04.877182 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:04.938202 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e07edb-d511-4ee4-a5db-c9ff2c551284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:04.938865 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e07edb-d511-4ee4-a5db-c9ff2c551284 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:04.939146 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e07edb-d511-4ee4-a5db-c9ff2c551284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:04.939413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e07edb-d511-4ee4-a5db-c9ff2c551284 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:04.945770 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e07edb-d511-4ee4-a5db-c9ff2c551284 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:04.947883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e07edb-d511-4ee4-a5db-c9ff2c551284 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:05.005433 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e2e77d-54c9-4166-9e4d-07b0fbe2bf15 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:05.016399 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713f517-e492-4b22-a9eb-854999493625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:05.017268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713f517-e492-4b22-a9eb-854999493625 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:05.017800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713f517-e492-4b22-a9eb-854999493625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:05.018435 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713f517-e492-4b22-a9eb-854999493625 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:05.019685 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713f517-e492-4b22-a9eb-854999493625 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:05.028057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713f517-e492-4b22-a9eb-854999493625 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:05.089515 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 851/1689] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:00:04 2022] POST /identity/v3/projects => generated 405 bytes in 457 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:00:05.091003 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e07edb-d511-4ee4-a5db-c9ff2c551284 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:05.114332 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 839/1690] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:00:04 2022] POST /identity/v3/projects => generated 363 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:05.115453 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713f517-e492-4b22-a9eb-854999493625 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:05.279841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e07edb-d511-4ee4-a5db-c9ff2c551284 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:05.349975 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713f517-e492-4b22-a9eb-854999493625 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:05.364785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97436a4-2e36-48e0-b534-c1784e8ad51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:05.365691 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97436a4-2e36-48e0-b534-c1784e8ad51a admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:05.365982 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97436a4-2e36-48e0-b534-c1784e8ad51a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:05.366258 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97436a4-2e36-48e0-b534-c1784e8ad51a admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:05.366975 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97436a4-2e36-48e0-b534-c1784e8ad51a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:05.369130 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97436a4-2e36-48e0-b534-c1784e8ad51a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:05.403705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:05.421505 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 840/1691] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:00:05 2022] POST /identity/v3/users => generated 288 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:05.431819 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97436a4-2e36-48e0-b534-c1784e8ad51a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:05.456172 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97436a4-2e36-48e0-b534-c1784e8ad51a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:05.463394 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978347e1-f2e3-4b40-ba9f-32f8d9afe3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:05.464312 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978347e1-f2e3-4b40-ba9f-32f8d9afe3ba admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:05.464856 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978347e1-f2e3-4b40-ba9f-32f8d9afe3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:05.465013 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978347e1-f2e3-4b40-ba9f-32f8d9afe3ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:05.465981 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978347e1-f2e3-4b40-ba9f-32f8d9afe3ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:05.467801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978347e1-f2e3-4b40-ba9f-32f8d9afe3ba admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:05.501731 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 841/1692] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:00:05.506038 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978347e1-f2e3-4b40-ba9f-32f8d9afe3ba admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:05.506371 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedcc9df-eb70-4595-952f-59b17a116284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:05.507064 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcc9df-eb70-4595-952f-59b17a116284 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:05.507576 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcc9df-eb70-4595-952f-59b17a116284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:05.507748 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcc9df-eb70-4595-952f-59b17a116284 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:05.508413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedcc9df-eb70-4595-952f-59b17a116284 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:05.510772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedcc9df-eb70-4595-952f-59b17a116284 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:05.525343 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978347e1-f2e3-4b40-ba9f-32f8d9afe3ba admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:05.536965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a060710-3753-4ad7-85d7-3a5c9f13d1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:05.537687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a060710-3753-4ad7-85d7-3a5c9f13d1cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:05.537989 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a060710-3753-4ad7-85d7-3a5c9f13d1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:05.538271 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a060710-3753-4ad7-85d7-3a5c9f13d1cb admin admin] PATH_INFO: `/v3/projects/6c17ed3b3b9945da987db43b817faba1/users/8736ccfd969f41aa8adf822c2bb32910/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:05.562706 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 852/1693] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:00:05 2022] POST /identity/v3/users => generated 309 bytes in 468 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:05.564347 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedcc9df-eb70-4595-952f-59b17a116284 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:05.578200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a060710-3753-4ad7-85d7-3a5c9f13d1cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c17ed3b3b9945da987db43b817faba1, user_id=8736ccfd969f41aa8adf822c2bb32910, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:05.582207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedcc9df-eb70-4595-952f-59b17a116284 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:05.582562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a060710-3753-4ad7-85d7-3a5c9f13d1cb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:05.595872 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c435b4-b988-45de-9125-9c83bc83f47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:05.596528 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c435b4-b988-45de-9125-9c83bc83f47f admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:05.596810 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c435b4-b988-45de-9125-9c83bc83f47f admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:05.597085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c435b4-b988-45de-9125-9c83bc83f47f admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:05.597994 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c435b4-b988-45de-9125-9c83bc83f47f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:05.599915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c435b4-b988-45de-9125-9c83bc83f47f admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:05.618919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 842/1694] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:05 2022] PUT /identity/v3/projects/6c17ed3b3b9945da987db43b817faba1/users/8736ccfd969f41aa8adf822c2bb32910/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:05.623928 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a060710-3753-4ad7-85d7-3a5c9f13d1cb admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:05.639077 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 853/1695] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:00:05.648778 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c435b4-b988-45de-9125-9c83bc83f47f admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:05.672098 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c435b4-b988-45de-9125-9c83bc83f47f admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:05.790306 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a060710-3753-4ad7-85d7-3a5c9f13d1cb admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:05.812811 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae17b1-c125-4ba3-ad6b-c2bfca515828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:05.813487 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae17b1-c125-4ba3-ad6b-c2bfca515828 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:05.813771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae17b1-c125-4ba3-ad6b-c2bfca515828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:05.814047 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae17b1-c125-4ba3-ad6b-c2bfca515828 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:05.814768 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae17b1-c125-4ba3-ad6b-c2bfca515828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:05.817015 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae17b1-c125-4ba3-ad6b-c2bfca515828 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:05.826140 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c470749-4fee-4274-a274-12349914e0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:05.827378 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c470749-4fee-4274-a274-12349914e0a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:05.829205 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c470749-4fee-4274-a274-12349914e0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:05.829721 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c470749-4fee-4274-a274-12349914e0a2 admin admin] PATH_INFO: `/v3/projects/f50bae84e5294d469fc867a01cdb2602/users/7acc033a5bcc4cabb12bf180e7a31f89/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:05.839274 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:05.850022 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 843/1696] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:05.867073 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae17b1-c125-4ba3-ad6b-c2bfca515828 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:05.894587 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c470749-4fee-4274-a274-12349914e0a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f50bae84e5294d469fc867a01cdb2602, user_id=7acc033a5bcc4cabb12bf180e7a31f89, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:05.897999 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c470749-4fee-4274-a274-12349914e0a2 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:05.900116 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae17b1-c125-4ba3-ad6b-c2bfca515828 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:05.905848 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77416cb1-9ec2-482a-91bd-ab40d1e3ab58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:05.906591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77416cb1-9ec2-482a-91bd-ab40d1e3ab58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:05.906880 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77416cb1-9ec2-482a-91bd-ab40d1e3ab58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:05.907179 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77416cb1-9ec2-482a-91bd-ab40d1e3ab58 admin admin] PATH_INFO: `/v3/projects/6c17ed3b3b9945da987db43b817faba1/users/8736ccfd969f41aa8adf822c2bb32910/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:05.919584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77416cb1-9ec2-482a-91bd-ab40d1e3ab58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c17ed3b3b9945da987db43b817faba1, user_id=8736ccfd969f41aa8adf822c2bb32910, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:05.922571 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77416cb1-9ec2-482a-91bd-ab40d1e3ab58 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:05.927138 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 854/1697] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:05 2022] PUT /identity/v3/projects/f50bae84e5294d469fc867a01cdb2602/users/7acc033a5bcc4cabb12bf180e7a31f89/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:05.929333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c470749-4fee-4274-a274-12349914e0a2 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:05.944700 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 844/1698] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:05 2022] PUT /identity/v3/projects/6c17ed3b3b9945da987db43b817faba1/users/8736ccfd969f41aa8adf822c2bb32910/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:05.958352 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac9f86-ecab-4517-a57b-8c1bbb1e5709 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:05.958684 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac9f86-ecab-4517-a57b-8c1bbb1e5709 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:05.959051 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac9f86-ecab-4517-a57b-8c1bbb1e5709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:06.032015 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2eac9f86-ecab-4517-a57b-8c1bbb1e5709 None None] MFA Rules not processed for user `8736ccfd969f41aa8adf822c2bb32910`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:06.033281 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2eac9f86-ecab-4517-a57b-8c1bbb1e5709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:06.057887 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c470749-4fee-4274-a274-12349914e0a2 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:06.254856 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 845/1699] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:05 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:06.264219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee394c-1bcc-4f51-ab77-934ad02a400e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:06.265091 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee394c-1bcc-4f51-ab77-934ad02a400e admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:06.265217 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee394c-1bcc-4f51-ab77-934ad02a400e admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:06.265406 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee394c-1bcc-4f51-ab77-934ad02a400e admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:06.266114 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee394c-1bcc-4f51-ab77-934ad02a400e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:06.267934 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee394c-1bcc-4f51-ab77-934ad02a400e admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:06.311463 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac9f86-ecab-4517-a57b-8c1bbb1e5709 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:06.322542 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 855/1700] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:06.325307 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee394c-1bcc-4f51-ab77-934ad02a400e admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:06.326745 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac9f86-ecab-4517-a57b-8c1bbb1e5709 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:06.354237 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9cb457-3bc1-4de1-9c7a-e54b387e2fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:06.354919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cb457-3bc1-4de1-9c7a-e54b387e2fc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:06.355143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cb457-3bc1-4de1-9c7a-e54b387e2fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:06.355412 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cb457-3bc1-4de1-9c7a-e54b387e2fc7 admin admin] PATH_INFO: `/v3/projects/f50bae84e5294d469fc867a01cdb2602/users/7acc033a5bcc4cabb12bf180e7a31f89/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:06.368440 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9cb457-3bc1-4de1-9c7a-e54b387e2fc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f50bae84e5294d469fc867a01cdb2602, user_id=7acc033a5bcc4cabb12bf180e7a31f89, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:06.371424 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9cb457-3bc1-4de1-9c7a-e54b387e2fc7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:06.412151 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 846/1701] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:06 2022] PUT /identity/v3/projects/f50bae84e5294d469fc867a01cdb2602/users/7acc033a5bcc4cabb12bf180e7a31f89/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:00:06.466637 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee394c-1bcc-4f51-ab77-934ad02a400e admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:06.570914 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:06.597970 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74031715-a883-46d5-8f1e-07767cd1432d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:06.598935 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74031715-a883-46d5-8f1e-07767cd1432d service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:06.599365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74031715-a883-46d5-8f1e-07767cd1432d service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:06.599841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74031715-a883-46d5-8f1e-07767cd1432d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:06.732877 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74031715-a883-46d5-8f1e-07767cd1432d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:06.755119 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74031715-a883-46d5-8f1e-07767cd1432d service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:06.783935 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71585a33-c337-4d91-ad63-adca8180e43b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:06.784312 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71585a33-c337-4d91-ad63-adca8180e43b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:06.784566 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71585a33-c337-4d91-ad63-adca8180e43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:06.802924 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 856/1702] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:06 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:06.845123 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-71585a33-c337-4d91-ad63-adca8180e43b None None] MFA Rules not processed for user `f996bed3f6064c6dbf04d26c7c04df7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:06.849348 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-71585a33-c337-4d91-ad63-adca8180e43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:06.864882 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:06.992834 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 847/1703] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:07.009867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74031715-a883-46d5-8f1e-07767cd1432d service cinder] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:07.013261 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71585a33-c337-4d91-ad63-adca8180e43b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:07.014927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-71585a33-c337-4d91-ad63-adca8180e43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:07.021802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74031715-a883-46d5-8f1e-07767cd1432d service cinder] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:07.032210 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faada319-ae4b-4251-b7a6-827d30822ea4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:07.033120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faada319-ae4b-4251-b7a6-827d30822ea4 service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:07.033653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faada319-ae4b-4251-b7a6-827d30822ea4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:07.034132 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faada319-ae4b-4251-b7a6-827d30822ea4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:07.151434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faada319-ae4b-4251-b7a6-827d30822ea4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:07.160959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faada319-ae4b-4251-b7a6-827d30822ea4 service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:07.185132 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 857/1704] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:07 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:07.208577 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71585a33-c337-4d91-ad63-adca8180e43b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:07.254097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900989d-bcdb-4187-8c69-8d067c5930ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:07.254917 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900989d-bcdb-4187-8c69-8d067c5930ef service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:07.255140 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900989d-bcdb-4187-8c69-8d067c5930ef service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:07.255453 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900989d-bcdb-4187-8c69-8d067c5930ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:07.282532 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900989d-bcdb-4187-8c69-8d067c5930ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:07.284431 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900989d-bcdb-4187-8c69-8d067c5930ef service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:07.329608 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 848/1705] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:00:07.588245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faada319-ae4b-4251-b7a6-827d30822ea4 service cinder] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:07.590111 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-faada319-ae4b-4251-b7a6-827d30822ea4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:07.695617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900989d-bcdb-4187-8c69-8d067c5930ef service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:07.699937 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3900989d-bcdb-4187-8c69-8d067c5930ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:07.821789 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faada319-ae4b-4251-b7a6-827d30822ea4 service cinder] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:07.836681 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c979c1-e15f-4cd3-992e-6b1b7329ec11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:07.840187 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c979c1-e15f-4cd3-992e-6b1b7329ec11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:07.841383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c979c1-e15f-4cd3-992e-6b1b7329ec11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:07.843906 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c979c1-e15f-4cd3-992e-6b1b7329ec11 admin admin] PATH_INFO: `/v3/users/44065be4a0c541289827f5586aa43c69` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:07.849026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c979c1-e15f-4cd3-992e-6b1b7329ec11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44065be4a0c541289827f5586aa43c69)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:07.854822 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c979c1-e15f-4cd3-992e-6b1b7329ec11 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:07.938037 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900989d-bcdb-4187-8c69-8d067c5930ef service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:07.945445 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022bb63e-8410-489a-83bd-f2db63bf3b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:07.946174 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:07.949166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bb63e-8410-489a-83bd-f2db63bf3b55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:07.952091 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bb63e-8410-489a-83bd-f2db63bf3b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:07.953263 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bb63e-8410-489a-83bd-f2db63bf3b55 admin admin] PATH_INFO: `/v3/users/154ade73bc7a40458078170cc73d0634` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:07.955121 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022bb63e-8410-489a-83bd-f2db63bf3b55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=154ade73bc7a40458078170cc73d0634)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:07.978077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022bb63e-8410-489a-83bd-f2db63bf3b55 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:07.979856 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-04c979c1-e15f-4cd3-992e-6b1b7329ec11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44065be4a0c541289827f5586aa43c69'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:07.999374 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-04c979c1-e15f-4cd3-992e-6b1b7329ec11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44065be4a0c541289827f5586aa43c69'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:08.024812 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-04c979c1-e15f-4cd3-992e-6b1b7329ec11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44065be4a0c541289827f5586aa43c69'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:08.025666 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-04c979c1-e15f-4cd3-992e-6b1b7329ec11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44065be4a0c541289827f5586aa43c69'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:08.026488 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-04c979c1-e15f-4cd3-992e-6b1b7329ec11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44065be4a0c541289827f5586aa43c69'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:08.068138 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 858/1706] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:00:07 2022] DELETE /identity/v3/users/44065be4a0c541289827f5586aa43c69 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:08.068606 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:08.074364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b487d-43a6-4ae1-a2c4-6fcc2520f46f None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:08.074650 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b487d-43a6-4ae1-a2c4-6fcc2520f46f None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:08.074940 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b487d-43a6-4ae1-a2c4-6fcc2520f46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:08.099022 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-022bb63e-8410-489a-83bd-f2db63bf3b55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '154ade73bc7a40458078170cc73d0634'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:08.110466 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-022bb63e-8410-489a-83bd-f2db63bf3b55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '154ade73bc7a40458078170cc73d0634'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:08.111906 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-022bb63e-8410-489a-83bd-f2db63bf3b55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '154ade73bc7a40458078170cc73d0634'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:08.129907 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-022bb63e-8410-489a-83bd-f2db63bf3b55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '154ade73bc7a40458078170cc73d0634'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:08.131536 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-022bb63e-8410-489a-83bd-f2db63bf3b55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '154ade73bc7a40458078170cc73d0634'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:08.140166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-421b487d-43a6-4ae1-a2c4-6fcc2520f46f None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:08.142728 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-421b487d-43a6-4ae1-a2c4-6fcc2520f46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:08.179590 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 849/1707] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:00:07 2022] DELETE /identity/v3/users/154ade73bc7a40458078170cc73d0634 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:08.184423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53effe6a-84a7-44fd-8c0d-d5892511c418 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:08.184812 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53effe6a-84a7-44fd-8c0d-d5892511c418 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:08.185068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53effe6a-84a7-44fd-8c0d-d5892511c418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:08.265684 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-53effe6a-84a7-44fd-8c0d-d5892511c418 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:08.267725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-53effe6a-84a7-44fd-8c0d-d5892511c418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:08.308126 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 859/1708] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:08.326527 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b487d-43a6-4ae1-a2c4-6fcc2520f46f None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:08.328282 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-421b487d-43a6-4ae1-a2c4-6fcc2520f46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:08.401367 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 850/1709] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:08.419962 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53effe6a-84a7-44fd-8c0d-d5892511c418 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:08.422831 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-53effe6a-84a7-44fd-8c0d-d5892511c418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:08.490978 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b487d-43a6-4ae1-a2c4-6fcc2520f46f None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:08.612563 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53effe6a-84a7-44fd-8c0d-d5892511c418 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:08.625695 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27e11a-1dfa-4a8a-a08a-c7ae6b2b5a81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:08.626644 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27e11a-1dfa-4a8a-a08a-c7ae6b2b5a81 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:08.626993 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27e11a-1dfa-4a8a-a08a-c7ae6b2b5a81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:08.627331 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27e11a-1dfa-4a8a-a08a-c7ae6b2b5a81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:08.659748 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27e11a-1dfa-4a8a-a08a-c7ae6b2b5a81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:08.665859 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6f17b-cee6-40a6-86c6-a00cbace5bfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:08.666590 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27e11a-1dfa-4a8a-a08a-c7ae6b2b5a81 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:08.673074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6f17b-cee6-40a6-86c6-a00cbace5bfb service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:08.673436 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6f17b-cee6-40a6-86c6-a00cbace5bfb service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:08.673709 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6f17b-cee6-40a6-86c6-a00cbace5bfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:08.694370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 851/1710] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:00:08.696638 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee6f17b-cee6-40a6-86c6-a00cbace5bfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:08.698340 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee6f17b-cee6-40a6-86c6-a00cbace5bfb service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:08.711142 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 860/1711] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:00:10.258791 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27e11a-1dfa-4a8a-a08a-c7ae6b2b5a81 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:10.261815 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fc27e11a-1dfa-4a8a-a08a-c7ae6b2b5a81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:10.287759 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27e11a-1dfa-4a8a-a08a-c7ae6b2b5a81 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:10.302886 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d9b408-ef6c-42ce-a23c-50dfe0decb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:10.303609 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9b408-ef6c-42ce-a23c-50dfe0decb99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:10.303896 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9b408-ef6c-42ce-a23c-50dfe0decb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:10.304175 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9b408-ef6c-42ce-a23c-50dfe0decb99 admin admin] PATH_INFO: `/v3/projects/c295b736057e4f73b562a5c41f9717dd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:10.309676 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d9b408-ef6c-42ce-a23c-50dfe0decb99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c295b736057e4f73b562a5c41f9717dd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:10.312591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d9b408-ef6c-42ce-a23c-50dfe0decb99 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:10.631997 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-98d9b408-ef6c-42ce-a23c-50dfe0decb99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c295b736057e4f73b562a5c41f9717dd', 'request_id': 'req-98d9b408-ef6c-42ce-a23c-50dfe0decb99'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:10.638480 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:10.676527 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6f17b-cee6-40a6-86c6-a00cbace5bfb service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:10.679299 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-6ee6f17b-cee6-40a6-86c6-a00cbace5bfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:10.695184 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-98d9b408-ef6c-42ce-a23c-50dfe0decb99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c295b736057e4f73b562a5c41f9717dd', 'request_id': 'req-98d9b408-ef6c-42ce-a23c-50dfe0decb99'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:10.713935 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-98d9b408-ef6c-42ce-a23c-50dfe0decb99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c295b736057e4f73b562a5c41f9717dd', 'request_id': 'req-98d9b408-ef6c-42ce-a23c-50dfe0decb99'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:10.735724 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-98d9b408-ef6c-42ce-a23c-50dfe0decb99 admin admin] The token cache is being invalidate because project c295b736057e4f73b562a5c41f9717dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:00:10.737988 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-98d9b408-ef6c-42ce-a23c-50dfe0decb99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:10.739871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-98d9b408-ef6c-42ce-a23c-50dfe0decb99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:10.744596 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 852/1712] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:00:10 2022] DELETE /identity/v3/projects/c295b736057e4f73b562a5c41f9717dd => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:10.752129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a45a26-c3cb-4fcf-b8f3-b51354b72b8a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:10.752694 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a45a26-c3cb-4fcf-b8f3-b51354b72b8a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:10.753232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a45a26-c3cb-4fcf-b8f3-b51354b72b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:10.759368 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6f17b-cee6-40a6-86c6-a00cbace5bfb service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:10.787712 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:10.846854 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d7a45a26-c3cb-4fcf-b8f3-b51354b72b8a None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:10.848373 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d7a45a26-c3cb-4fcf-b8f3-b51354b72b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:10.998345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759aab13-6128-4092-842e-a3cfb0a0e670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:10.999197 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759aab13-6128-4092-842e-a3cfb0a0e670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:11.000643 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759aab13-6128-4092-842e-a3cfb0a0e670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:11.001054 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759aab13-6128-4092-842e-a3cfb0a0e670 admin admin] PATH_INFO: `/v3/projects/88f35a8508a2419d84734748589ea81a` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:11.014560 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759aab13-6128-4092-842e-a3cfb0a0e670 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88f35a8508a2419d84734748589ea81a)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:11.021920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759aab13-6128-4092-842e-a3cfb0a0e670 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:11.072640 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 853/1713] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:11.079176 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f39f3f-6a85-4d70-a830-6f5a39fb1062 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:11.079884 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f39f3f-6a85-4d70-a830-6f5a39fb1062 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:11.080375 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f39f3f-6a85-4d70-a830-6f5a39fb1062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:11.132562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b3f39f3f-6a85-4d70-a830-6f5a39fb1062 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:11.136817 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b3f39f3f-6a85-4d70-a830-6f5a39fb1062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:11.177219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-759aab13-6128-4092-842e-a3cfb0a0e670 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88f35a8508a2419d84734748589ea81a', 'request_id': 'req-759aab13-6128-4092-842e-a3cfb0a0e670'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:11.205656 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-759aab13-6128-4092-842e-a3cfb0a0e670 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88f35a8508a2419d84734748589ea81a', 'request_id': 'req-759aab13-6128-4092-842e-a3cfb0a0e670'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:11.233693 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-759aab13-6128-4092-842e-a3cfb0a0e670 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88f35a8508a2419d84734748589ea81a', 'request_id': 'req-759aab13-6128-4092-842e-a3cfb0a0e670'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:11.285205 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-759aab13-6128-4092-842e-a3cfb0a0e670 admin admin] The token cache is being invalidate because project 88f35a8508a2419d84734748589ea81a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:00:11.286002 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-759aab13-6128-4092-842e-a3cfb0a0e670 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:11.287236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-759aab13-6128-4092-842e-a3cfb0a0e670 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:11.289361 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 861/1714] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:00:10 2022] DELETE /identity/v3/projects/88f35a8508a2419d84734748589ea81a => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:11.294244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759aab13-6128-4092-842e-a3cfb0a0e670 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:11.304507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-759aab13-6128-4092-842e-a3cfb0a0e670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:11.412579 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 854/1715] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:11.420778 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f39f3f-6a85-4d70-a830-6f5a39fb1062 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:11.451211 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f39f3f-6a85-4d70-a830-6f5a39fb1062 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:11.481396 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc86e42-937a-496d-a1f0-2afa420b5bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:11.482144 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc86e42-937a-496d-a1f0-2afa420b5bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:11.482352 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc86e42-937a-496d-a1f0-2afa420b5bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:11.482633 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc86e42-937a-496d-a1f0-2afa420b5bb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:11.483365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc86e42-937a-496d-a1f0-2afa420b5bb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:00:11.483704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc86e42-937a-496d-a1f0-2afa420b5bb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:11.485401 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc86e42-937a-496d-a1f0-2afa420b5bb4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:11.546036 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 855/1716] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:00:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:00:11.547097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc86e42-937a-496d-a1f0-2afa420b5bb4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:11.567064 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759aab13-6128-4092-842e-a3cfb0a0e670 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:11.571973 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc86e42-937a-496d-a1f0-2afa420b5bb4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:11.573415 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a77a1b-2b8f-45ee-8788-ba761a2416a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:11.574700 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a77a1b-2b8f-45ee-8788-ba761a2416a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:11.575384 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a77a1b-2b8f-45ee-8788-ba761a2416a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:11.575909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a77a1b-2b8f-45ee-8788-ba761a2416a7 admin admin] PATH_INFO: `/v3/users/5cec519e633e4a2ba70e0bac981cee6e` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:11.578847 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a77a1b-2b8f-45ee-8788-ba761a2416a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cec519e633e4a2ba70e0bac981cee6e)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:11.581199 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a77a1b-2b8f-45ee-8788-ba761a2416a7 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:11.586289 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325adbf-53fd-4291-995f-951ac9f606aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:11.586954 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325adbf-53fd-4291-995f-951ac9f606aa admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:11.587276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325adbf-53fd-4291-995f-951ac9f606aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:11.587943 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325adbf-53fd-4291-995f-951ac9f606aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:11.592211 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5325adbf-53fd-4291-995f-951ac9f606aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:11.600863 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5325adbf-53fd-4291-995f-951ac9f606aa admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:11.674465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 856/1717] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:00:11 2022] POST /identity/v3/projects => generated 387 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:11.676316 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325adbf-53fd-4291-995f-951ac9f606aa admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:11.696212 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:11.716979 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-e3a77a1b-2b8f-45ee-8788-ba761a2416a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cec519e633e4a2ba70e0bac981cee6e'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:11.736347 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-e3a77a1b-2b8f-45ee-8788-ba761a2416a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cec519e633e4a2ba70e0bac981cee6e'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:11.773624 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-e3a77a1b-2b8f-45ee-8788-ba761a2416a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cec519e633e4a2ba70e0bac981cee6e'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:11.775632 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-e3a77a1b-2b8f-45ee-8788-ba761a2416a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cec519e633e4a2ba70e0bac981cee6e'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:11.777384 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-e3a77a1b-2b8f-45ee-8788-ba761a2416a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cec519e633e4a2ba70e0bac981cee6e'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:11.815333 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:11.863293 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 862/1718] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:00:11 2022] DELETE /identity/v3/users/5cec519e633e4a2ba70e0bac981cee6e => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:11.865137 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325adbf-53fd-4291-995f-951ac9f606aa admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:12.071452 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9d302-1855-4a0a-9589-f0e5a429fc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:12.085992 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9d302-1855-4a0a-9589-f0e5a429fc7c admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:12.088369 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9d302-1855-4a0a-9589-f0e5a429fc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:12.090456 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9d302-1855-4a0a-9589-f0e5a429fc7c admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:12.094544 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba9d302-1855-4a0a-9589-f0e5a429fc7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:12.118701 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba9d302-1855-4a0a-9589-f0e5a429fc7c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:12.230194 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 857/1719] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:00:11 2022] POST /identity/v3/users => generated 294 bytes in 555 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:00:12.235903 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a77a1b-2b8f-45ee-8788-ba761a2416a7 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:12.237447 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-e3a77a1b-2b8f-45ee-8788-ba761a2416a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:12.262481 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a77a1b-2b8f-45ee-8788-ba761a2416a7 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:12.290154 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7ea0d-0c68-4a90-8f0e-77269c3b8234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:12.292653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7ea0d-0c68-4a90-8f0e-77269c3b8234 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:12.292949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7ea0d-0c68-4a90-8f0e-77269c3b8234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:12.293229 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7ea0d-0c68-4a90-8f0e-77269c3b8234 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:12.294083 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df7ea0d-0c68-4a90-8f0e-77269c3b8234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:12.301489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df7ea0d-0c68-4a90-8f0e-77269c3b8234 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:12.358226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9d302-1855-4a0a-9589-f0e5a429fc7c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:12.360217 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 863/1720] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:12 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:00:12.384898 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9d302-1855-4a0a-9589-f0e5a429fc7c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:12.416931 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cec1a-6dfb-49d5-95c4-be6860e173ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:12.417741 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cec1a-6dfb-49d5-95c4-be6860e173ae admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:12.417741 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cec1a-6dfb-49d5-95c4-be6860e173ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:12.417741 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cec1a-6dfb-49d5-95c4-be6860e173ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:12.418406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8cec1a-6dfb-49d5-95c4-be6860e173ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:12.435208 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8cec1a-6dfb-49d5-95c4-be6860e173ae admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:12.452127 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79d4df-0b11-4e54-a6e1-e3baea3e2c91 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:12.452837 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79d4df-0b11-4e54-a6e1-e3baea3e2c91 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:12.453127 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79d4df-0b11-4e54-a6e1-e3baea3e2c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:12.473697 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 858/1721] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:12 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:12.484352 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cec1a-6dfb-49d5-95c4-be6860e173ae admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:12.521751 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-ed79d4df-0b11-4e54-a6e1-e3baea3e2c91 None None] MFA Rules not processed for user `f996bed3f6064c6dbf04d26c7c04df7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:12.521751 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-ed79d4df-0b11-4e54-a6e1-e3baea3e2c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:12.522580 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cec1a-6dfb-49d5-95c4-be6860e173ae admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:12.540038 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be65a86-3b68-4ce6-aeba-9f5320004cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:12.540777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be65a86-3b68-4ce6-aeba-9f5320004cce admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:12.541066 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be65a86-3b68-4ce6-aeba-9f5320004cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:12.544779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be65a86-3b68-4ce6-aeba-9f5320004cce admin admin] PATH_INFO: `/v3/projects/fc0d4aa379034eb7839167e0ee119be3/users/4e2d16ed40974a5db3d64f7dbe067344/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:12.603214 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be65a86-3b68-4ce6-aeba-9f5320004cce admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc0d4aa379034eb7839167e0ee119be3, user_id=4e2d16ed40974a5db3d64f7dbe067344, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:12.613735 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be65a86-3b68-4ce6-aeba-9f5320004cce admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:12.645606 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 859/1722] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:12 2022] PUT /identity/v3/projects/fc0d4aa379034eb7839167e0ee119be3/users/4e2d16ed40974a5db3d64f7dbe067344/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:12.662260 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 864/1723] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:00:12.671312 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2502b97-dcd6-47af-af5a-6ecb50e3389b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:12.673487 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2502b97-dcd6-47af-af5a-6ecb50e3389b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:12.674070 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2502b97-dcd6-47af-af5a-6ecb50e3389b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:12.759985 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e2502b97-dcd6-47af-af5a-6ecb50e3389b None None] MFA Rules not processed for user `4e2d16ed40974a5db3d64f7dbe067344`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:12.785949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252ae7e-2bd4-4135-8713-72ab815350e5 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:12.786729 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252ae7e-2bd4-4135-8713-72ab815350e5 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:12.787084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252ae7e-2bd4-4135-8713-72ab815350e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:12.791197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e2502b97-dcd6-47af-af5a-6ecb50e3389b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:12.895119 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-a252ae7e-2bd4-4135-8713-72ab815350e5 None None] MFA Rules not processed for user `39de5897bd2d4e2b91c05d8dda5827be`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:12.899691 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a252ae7e-2bd4-4135-8713-72ab815350e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:13.084084 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 860/1724] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:13.089291 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2502b97-dcd6-47af-af5a-6ecb50e3389b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:13.091473 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e2502b97-dcd6-47af-af5a-6ecb50e3389b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:13.224486 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 865/1725] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 439 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:13.225818 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252ae7e-2bd4-4135-8713-72ab815350e5 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:13.255007 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:13.328579 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2502b97-dcd6-47af-af5a-6ecb50e3389b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:13.353776 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d16f5-efd1-4083-8e0f-17190278d632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:13.359107 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d16f5-efd1-4083-8e0f-17190278d632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:13.359432 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d16f5-efd1-4083-8e0f-17190278d632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:13.359784 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d16f5-efd1-4083-8e0f-17190278d632 admin admin] PATH_INFO: `/v3/projects/7ac0576b99a24d5997072f864a86c8c0` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:13.362715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056d16f5-efd1-4083-8e0f-17190278d632 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ac0576b99a24d5997072f864a86c8c0)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:13.365082 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:13.366680 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056d16f5-efd1-4083-8e0f-17190278d632 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:13.423378 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252ae7e-2bd4-4135-8713-72ab815350e5 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:13.442906 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e6a82-a6e4-46a8-8df0-56a926d587db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:13.443583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e6a82-a6e4-46a8-8df0-56a926d587db admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:13.443870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e6a82-a6e4-46a8-8df0-56a926d587db admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:13.444145 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e6a82-a6e4-46a8-8df0-56a926d587db admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:13.444850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e6a82-a6e4-46a8-8df0-56a926d587db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:13.447933 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e6a82-a6e4-46a8-8df0-56a926d587db admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:13.519321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-056d16f5-efd1-4083-8e0f-17190278d632 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ac0576b99a24d5997072f864a86c8c0', 'request_id': 'req-056d16f5-efd1-4083-8e0f-17190278d632'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:13.530821 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 866/1726] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:00:13 2022] POST /identity/v3/projects => generated 385 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:13.530821 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e6a82-a6e4-46a8-8df0-56a926d587db admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:13.534912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-3c5e6a82-a6e4-46a8-8df0-56a926d587db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:13.556009 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-056d16f5-efd1-4083-8e0f-17190278d632 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ac0576b99a24d5997072f864a86c8c0', 'request_id': 'req-056d16f5-efd1-4083-8e0f-17190278d632'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:13.596423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-056d16f5-efd1-4083-8e0f-17190278d632 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ac0576b99a24d5997072f864a86c8c0', 'request_id': 'req-056d16f5-efd1-4083-8e0f-17190278d632'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:13.657446 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-056d16f5-efd1-4083-8e0f-17190278d632 admin admin] The token cache is being invalidate because project 7ac0576b99a24d5997072f864a86c8c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:00:13.658125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-056d16f5-efd1-4083-8e0f-17190278d632 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:13.663649 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-056d16f5-efd1-4083-8e0f-17190278d632 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:13.671591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 861/1727] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:00:13 2022] DELETE /identity/v3/projects/7ac0576b99a24d5997072f864a86c8c0 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:13.677540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d16f5-efd1-4083-8e0f-17190278d632 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:13.681425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-056d16f5-efd1-4083-8e0f-17190278d632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:13.706096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e6a82-a6e4-46a8-8df0-56a926d587db admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:13.712600 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ebe16-6770-4410-b466-f04a0da73180 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:13.713555 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ebe16-6770-4410-b466-f04a0da73180 service nova] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:13.713939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ebe16-6770-4410-b466-f04a0da73180 service nova] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:13.714316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ebe16-6770-4410-b466-f04a0da73180 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:13.716537 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-731ebe16-6770-4410-b466-f04a0da73180 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:13.855775 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731ebe16-6770-4410-b466-f04a0da73180 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:13.861583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731ebe16-6770-4410-b466-f04a0da73180 service nova] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:13.889283 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 867/1728] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:13.890542 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee1258-5173-40dc-8245-1a7d6ba53f00 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:13.892451 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee1258-5173-40dc-8245-1a7d6ba53f00 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:13.892930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee1258-5173-40dc-8245-1a7d6ba53f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:13.931964 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-1aee1258-5173-40dc-8245-1a7d6ba53f00 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:13.933404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-1aee1258-5173-40dc-8245-1a7d6ba53f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:14.018243 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 868/1729] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:14.020473 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc17d5-c72d-4dc4-8c3a-11922a5a9ebb None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:14.020915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc17d5-c72d-4dc4-8c3a-11922a5a9ebb None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:14.021370 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc17d5-c72d-4dc4-8c3a-11922a5a9ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:14.022454 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d16f5-efd1-4083-8e0f-17190278d632 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:14.032109 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:14.046698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-befc17d5-c72d-4dc4-8c3a-11922a5a9ebb None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:14.049107 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-befc17d5-c72d-4dc4-8c3a-11922a5a9ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:14.064387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 869/1730] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:14.066008 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc17d5-c72d-4dc4-8c3a-11922a5a9ebb None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:14.066156 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec53bf-47ae-4032-9487-2ecfd5757e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:14.069737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec53bf-47ae-4032-9487-2ecfd5757e1a admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:14.070292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec53bf-47ae-4032-9487-2ecfd5757e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:14.070810 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec53bf-47ae-4032-9487-2ecfd5757e1a admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:14.074519 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ec53bf-47ae-4032-9487-2ecfd5757e1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:14.090081 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ec53bf-47ae-4032-9487-2ecfd5757e1a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:14.092578 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc17d5-c72d-4dc4-8c3a-11922a5a9ebb None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:14.108671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95178a3a-e87b-4a6b-a188-81b3a311ef13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:14.109857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95178a3a-e87b-4a6b-a188-81b3a311ef13 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:14.110286 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95178a3a-e87b-4a6b-a188-81b3a311ef13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:14.110715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95178a3a-e87b-4a6b-a188-81b3a311ef13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:14.111674 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95178a3a-e87b-4a6b-a188-81b3a311ef13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:00:14.113474 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95178a3a-e87b-4a6b-a188-81b3a311ef13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:14.116634 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95178a3a-e87b-4a6b-a188-81b3a311ef13 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:14.166210 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 870/1731] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:00:14.171905 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95178a3a-e87b-4a6b-a188-81b3a311ef13 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:14.193500 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 862/1732] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:00:13 2022] POST /identity/v3/users => generated 299 bytes in 521 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:14.207209 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95178a3a-e87b-4a6b-a188-81b3a311ef13 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:14.225392 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68445b0b-fcbe-431a-bcf6-b25cf0db44f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:14.226222 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68445b0b-fcbe-431a-bcf6-b25cf0db44f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:14.226628 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68445b0b-fcbe-431a-bcf6-b25cf0db44f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:14.227017 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68445b0b-fcbe-431a-bcf6-b25cf0db44f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:14.228131 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68445b0b-fcbe-431a-bcf6-b25cf0db44f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:14.229314 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec53bf-47ae-4032-9487-2ecfd5757e1a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:14.234882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68445b0b-fcbe-431a-bcf6-b25cf0db44f4 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:14.256173 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec53bf-47ae-4032-9487-2ecfd5757e1a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:14.267539 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b87c8-a1ef-4231-be45-4d002bf792e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:14.269035 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b87c8-a1ef-4231-be45-4d002bf792e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:14.269781 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b87c8-a1ef-4231-be45-4d002bf792e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:14.270517 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b87c8-a1ef-4231-be45-4d002bf792e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:14.272047 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b87c8-a1ef-4231-be45-4d002bf792e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:14.276633 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b87c8-a1ef-4231-be45-4d002bf792e9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:14.315394 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 871/1733] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:00:14 2022] POST /identity/v3/projects => generated 387 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:14.319264 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68445b0b-fcbe-431a-bcf6-b25cf0db44f4 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:14.326911 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 863/1734] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:14.349743 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b87c8-a1ef-4231-be45-4d002bf792e9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:14.382292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b87c8-a1ef-4231-be45-4d002bf792e9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:14.491884 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68445b0b-fcbe-431a-bcf6-b25cf0db44f4 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:14.521108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85914547-6c68-48cc-85a6-2a88926428f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:14.521801 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85914547-6c68-48cc-85a6-2a88926428f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:14.522087 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85914547-6c68-48cc-85a6-2a88926428f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:14.522358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85914547-6c68-48cc-85a6-2a88926428f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:14.523062 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85914547-6c68-48cc-85a6-2a88926428f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:14.525418 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:14.542956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85914547-6c68-48cc-85a6-2a88926428f1 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:14.589380 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843d5cc-d7d7-4716-b2f1-15444f2d2eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:14.590446 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843d5cc-d7d7-4716-b2f1-15444f2d2eba admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:14.590748 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843d5cc-d7d7-4716-b2f1-15444f2d2eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:14.591141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843d5cc-d7d7-4716-b2f1-15444f2d2eba admin admin] PATH_INFO: `/v3/projects/5de03239c4e24cc7bf6dec1cd2da2567/users/c1d1cb4c636b4445ab2d2fec500c6c53/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:14.611381 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 872/1735] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:00:14 2022] POST /identity/v3/users => generated 294 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:14.616904 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf430a9e-9e81-4541-bfe6-14db1b5be918 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:14.617108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf430a9e-9e81-4541-bfe6-14db1b5be918 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:14.617385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf430a9e-9e81-4541-bfe6-14db1b5be918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:14.650757 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843d5cc-d7d7-4716-b2f1-15444f2d2eba admin admin] RBAC: Authorizing `identity:create_grant(project_id=5de03239c4e24cc7bf6dec1cd2da2567, user_id=c1d1cb4c636b4445ab2d2fec500c6c53, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:14.664793 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843d5cc-d7d7-4716-b2f1-15444f2d2eba admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:14.711822 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 864/1736] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:14 2022] PUT /identity/v3/projects/5de03239c4e24cc7bf6dec1cd2da2567/users/c1d1cb4c636b4445ab2d2fec500c6c53/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:14.714888 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d296e72-b849-4ec0-9e69-0b601120bc47 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:14.716759 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d296e72-b849-4ec0-9e69-0b601120bc47 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:14.717691 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d296e72-b849-4ec0-9e69-0b601120bc47 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:14.723725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 865/1737] 10.222.0.17 () {54 vars in 912 bytes} [Tue Jul 26 05:00:14 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 05:00:14.725490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d296e72-b849-4ec0-9e69-0b601120bc47 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:14.726772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-bf430a9e-9e81-4541-bfe6-14db1b5be918 None None] MFA Rules not processed for user `7acc033a5bcc4cabb12bf180e7a31f89`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:14.729814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-bf430a9e-9e81-4541-bfe6-14db1b5be918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:14.920643 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d296e72-b849-4ec0-9e69-0b601120bc47 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:14.936446 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5e5e0-2de1-40c2-8929-eddfe66b2bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:14.938013 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5e5e0-2de1-40c2-8929-eddfe66b2bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:14.943274 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5e5e0-2de1-40c2-8929-eddfe66b2bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:14.943627 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 873/1738] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:14 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:14.944854 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5e5e0-2de1-40c2-8929-eddfe66b2bf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:14.946831 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf430a9e-9e81-4541-bfe6-14db1b5be918 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:14.947434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b5e5e0-2de1-40c2-8929-eddfe66b2bf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:14.949792 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b5e5e0-2de1-40c2-8929-eddfe66b2bf1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:14.991449 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf430a9e-9e81-4541-bfe6-14db1b5be918 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:15.008297 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 866/1739] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:15.014732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a5dba-a624-483e-88a2-9b0d0e4e4d58 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:15.016315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a5dba-a624-483e-88a2-9b0d0e4e4d58 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:15.020793 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2000619-f7d8-40a7-bbf1-60ad92d433f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:15.021783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a5dba-a624-483e-88a2-9b0d0e4e4d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:15.026245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2000619-f7d8-40a7-bbf1-60ad92d433f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:15.029502 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2000619-f7d8-40a7-bbf1-60ad92d433f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:15.030413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2000619-f7d8-40a7-bbf1-60ad92d433f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:15.036804 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2000619-f7d8-40a7-bbf1-60ad92d433f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:15.043087 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2000619-f7d8-40a7-bbf1-60ad92d433f1 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:15.106659 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 874/1740] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:15.108223 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2000619-f7d8-40a7-bbf1-60ad92d433f1 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:15.117349 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-bc7a5dba-a624-483e-88a2-9b0d0e4e4d58 None None] MFA Rules not processed for user `dbdb8997891241e9b2e829feb8023f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:15.118902 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bc7a5dba-a624-483e-88a2-9b0d0e4e4d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:15.148219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2000619-f7d8-40a7-bbf1-60ad92d433f1 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:15.160756 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f5a5e-30e5-4ecd-a879-9f49a2ce6cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:15.173669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f5a5e-30e5-4ecd-a879-9f49a2ce6cbd admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:15.174359 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f5a5e-30e5-4ecd-a879-9f49a2ce6cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:15.175053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f5a5e-30e5-4ecd-a879-9f49a2ce6cbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:15.176426 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694f5a5e-30e5-4ecd-a879-9f49a2ce6cbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:15.182552 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694f5a5e-30e5-4ecd-a879-9f49a2ce6cbd admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:15.223108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 875/1741] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:15.225411 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f5a5e-30e5-4ecd-a879-9f49a2ce6cbd admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:15.266661 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 867/1742] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jul 26 05:00:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:15.271151 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a5dba-a624-483e-88a2-9b0d0e4e4d58 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:15.274463 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f5a5e-30e5-4ecd-a879-9f49a2ce6cbd admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:15.289321 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be3c20-1fde-4506-81a5-c17b5e7b357e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:15.292088 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be3c20-1fde-4506-81a5-c17b5e7b357e admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:15.292973 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be3c20-1fde-4506-81a5-c17b5e7b357e admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:15.293745 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be3c20-1fde-4506-81a5-c17b5e7b357e admin admin] PATH_INFO: `/v3/projects/5de03239c4e24cc7bf6dec1cd2da2567/users/c1d1cb4c636b4445ab2d2fec500c6c53/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:15.299178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be3c20-1fde-4506-81a5-c17b5e7b357e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5de03239c4e24cc7bf6dec1cd2da2567, user_id=c1d1cb4c636b4445ab2d2fec500c6c53, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:15.309506 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be3c20-1fde-4506-81a5-c17b5e7b357e admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:15.318021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a5dba-a624-483e-88a2-9b0d0e4e4d58 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:15.333670 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701a7c6-e6bd-432d-a8d5-4cefcdd5f89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:15.339011 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701a7c6-e6bd-432d-a8d5-4cefcdd5f89f admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:15.339011 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701a7c6-e6bd-432d-a8d5-4cefcdd5f89f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:15.339011 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701a7c6-e6bd-432d-a8d5-4cefcdd5f89f admin admin] PATH_INFO: `/v3/projects/ce5a4b3e5a7f4c15b0b1ca7758779773/users/c6112f9979c94d5aa6f8756bd1335479/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:15.348112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 876/1743] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:15 2022] PUT /identity/v3/projects/5de03239c4e24cc7bf6dec1cd2da2567/users/c1d1cb4c636b4445ab2d2fec500c6c53/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:15.350747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be3c20-1fde-4506-81a5-c17b5e7b357e admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:15.362942 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-b4be3c20-1fde-4506-81a5-c17b5e7b357e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:15.376556 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:15.394625 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8701a7c6-e6bd-432d-a8d5-4cefcdd5f89f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce5a4b3e5a7f4c15b0b1ca7758779773, user_id=c6112f9979c94d5aa6f8756bd1335479, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:15.397595 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8701a7c6-e6bd-432d-a8d5-4cefcdd5f89f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:15.423144 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 868/1744] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:15 2022] PUT /identity/v3/projects/ce5a4b3e5a7f4c15b0b1ca7758779773/users/c6112f9979c94d5aa6f8756bd1335479/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:15.437817 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d6a31-3953-411e-91fb-9651adf1756d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:15.438291 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d6a31-3953-411e-91fb-9651adf1756d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:15.438458 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d6a31-3953-411e-91fb-9651adf1756d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:15.515332 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-320d6a31-3953-411e-91fb-9651adf1756d None None] MFA Rules not processed for user `c1d1cb4c636b4445ab2d2fec500c6c53`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:15.516725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-320d6a31-3953-411e-91fb-9651adf1756d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:15.585876 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be3c20-1fde-4506-81a5-c17b5e7b357e admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:15.589376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:15.685146 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d83a34-88ba-4c95-b79d-b8e08d0f102d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:15.686135 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d83a34-88ba-4c95-b79d-b8e08d0f102d service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:15.686135 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d83a34-88ba-4c95-b79d-b8e08d0f102d service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:15.686432 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d83a34-88ba-4c95-b79d-b8e08d0f102d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:15.697073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d83a34-88ba-4c95-b79d-b8e08d0f102d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:15.698939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d83a34-88ba-4c95-b79d-b8e08d0f102d service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:15.701463 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 869/1745] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:15.706742 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a875878-9c50-4978-ac2a-67e7de79d1bb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:15.707911 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a875878-9c50-4978-ac2a-67e7de79d1bb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:15.710513 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a875878-9c50-4978-ac2a-67e7de79d1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:15.714629 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 877/1746] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:15.717105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d83a34-88ba-4c95-b79d-b8e08d0f102d service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:15.719304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-51d83a34-88ba-4c95-b79d-b8e08d0f102d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:15.832718 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9a875878-9c50-4978-ac2a-67e7de79d1bb None None] MFA Rules not processed for user `c6112f9979c94d5aa6f8756bd1335479`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:15.835984 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9a875878-9c50-4978-ac2a-67e7de79d1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:15.845274 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d83a34-88ba-4c95-b79d-b8e08d0f102d service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:15.854857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e7e21-f221-4544-a4cb-c069f5d128cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:15.857930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e7e21-f221-4544-a4cb-c069f5d128cd service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:15.858294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e7e21-f221-4544-a4cb-c069f5d128cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:15.858643 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e7e21-f221-4544-a4cb-c069f5d128cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:15.887263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e7e21-f221-4544-a4cb-c069f5d128cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:15.889324 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e7e21-f221-4544-a4cb-c069f5d128cd service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:15.924973 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 878/1747] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:15 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:16.014673 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e7e21-f221-4544-a4cb-c069f5d128cd service cinder] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:16.067965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 870/1748] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:16.073598 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a875878-9c50-4978-ac2a-67e7de79d1bb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:16.196972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e7e21-f221-4544-a4cb-c069f5d128cd service cinder] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:16.212839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a875878-9c50-4978-ac2a-67e7de79d1bb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:16.215773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d48df9-fa56-46a4-a9d3-4086a4afe544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:16.216461 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d48df9-fa56-46a4-a9d3-4086a4afe544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:16.216743 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d48df9-fa56-46a4-a9d3-4086a4afe544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:16.217013 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d48df9-fa56-46a4-a9d3-4086a4afe544 admin admin] PATH_INFO: `/v3/users/4e2d16ed40974a5db3d64f7dbe067344` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:16.218807 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d48df9-fa56-46a4-a9d3-4086a4afe544 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e2d16ed40974a5db3d64f7dbe067344)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:16.224117 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d48df9-fa56-46a4-a9d3-4086a4afe544 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:16.227866 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7204e17-8370-4dc2-93a5-948c02d94b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:16.230046 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7204e17-8370-4dc2-93a5-948c02d94b74 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:16.230602 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7204e17-8370-4dc2-93a5-948c02d94b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:16.230930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7204e17-8370-4dc2-93a5-948c02d94b74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:16.231676 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7204e17-8370-4dc2-93a5-948c02d94b74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:16.240520 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7204e17-8370-4dc2-93a5-948c02d94b74 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:16.242703 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:16.323600 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 871/1749] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:00:16 2022] POST /identity/v3/projects => generated 385 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:16.324607 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7204e17-8370-4dc2-93a5-948c02d94b74 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:16.388572 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:16.392428 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-77d48df9-fa56-46a4-a9d3-4086a4afe544 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e2d16ed40974a5db3d64f7dbe067344'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:16.416291 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-77d48df9-fa56-46a4-a9d3-4086a4afe544 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e2d16ed40974a5db3d64f7dbe067344'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:16.425446 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7204e17-8370-4dc2-93a5-948c02d94b74 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:16.443823 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-77d48df9-fa56-46a4-a9d3-4086a4afe544 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e2d16ed40974a5db3d64f7dbe067344'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:16.445496 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-77d48df9-fa56-46a4-a9d3-4086a4afe544 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e2d16ed40974a5db3d64f7dbe067344'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:16.446906 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-77d48df9-fa56-46a4-a9d3-4086a4afe544 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e2d16ed40974a5db3d64f7dbe067344'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:16.461571 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f7b7a-d986-4d15-b4c9-28ff98c259a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:16.467182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f7b7a-d986-4d15-b4c9-28ff98c259a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:16.467762 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f7b7a-d986-4d15-b4c9-28ff98c259a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:16.469842 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f7b7a-d986-4d15-b4c9-28ff98c259a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:16.474388 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f7b7a-d986-4d15-b4c9-28ff98c259a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:16.482504 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f7b7a-d986-4d15-b4c9-28ff98c259a7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:16.533409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 879/1750] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:00:16 2022] DELETE /identity/v3/users/4e2d16ed40974a5db3d64f7dbe067344 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:16.539614 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb31d1b-3a7f-4fee-9c15-9b7804a48015 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:16.539912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb31d1b-3a7f-4fee-9c15-9b7804a48015 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:16.545304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb31d1b-3a7f-4fee-9c15-9b7804a48015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:16.585758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 872/1751] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:00:16 2022] POST /identity/v3/users => generated 293 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:16.590362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f7b7a-d986-4d15-b4c9-28ff98c259a7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:16.597281 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-490f7b7a-d986-4d15-b4c9-28ff98c259a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:16.632161 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-abb31d1b-3a7f-4fee-9c15-9b7804a48015 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:16.636508 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-abb31d1b-3a7f-4fee-9c15-9b7804a48015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:16.650733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f7b7a-d986-4d15-b4c9-28ff98c259a7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:16.892393 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 880/1752] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:16.910060 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a77d9-8363-4d15-9b3d-c020f8eb5a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:16.910845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a77d9-8363-4d15-9b3d-c020f8eb5a29 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:16.911269 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a77d9-8363-4d15-9b3d-c020f8eb5a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:16.911716 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a77d9-8363-4d15-9b3d-c020f8eb5a29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:16.912605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a77d9-8363-4d15-9b3d-c020f8eb5a29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:16.914785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a77d9-8363-4d15-9b3d-c020f8eb5a29 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:16.930996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb31d1b-3a7f-4fee-9c15-9b7804a48015 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:16.932555 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-abb31d1b-3a7f-4fee-9c15-9b7804a48015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:16.947321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 873/1753] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:16.953446 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a77d9-8363-4d15-9b3d-c020f8eb5a29 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:17.062607 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a77d9-8363-4d15-9b3d-c020f8eb5a29 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:17.081139 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a0aa19-cf35-4188-b9a6-e5a0e281f84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:17.083866 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0aa19-cf35-4188-b9a6-e5a0e281f84a admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:17.083866 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0aa19-cf35-4188-b9a6-e5a0e281f84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:17.084278 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0aa19-cf35-4188-b9a6-e5a0e281f84a admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:17.090494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb31d1b-3a7f-4fee-9c15-9b7804a48015 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:17.093172 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a0aa19-cf35-4188-b9a6-e5a0e281f84a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:17.094814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34232a1a-5e8d-4c31-9bc2-ef38a40170b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:17.095760 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34232a1a-5e8d-4c31-9bc2-ef38a40170b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:17.096217 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34232a1a-5e8d-4c31-9bc2-ef38a40170b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:17.096634 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34232a1a-5e8d-4c31-9bc2-ef38a40170b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:17.100966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a0aa19-cf35-4188-b9a6-e5a0e281f84a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:17.119067 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34232a1a-5e8d-4c31-9bc2-ef38a40170b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:17.123060 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34232a1a-5e8d-4c31-9bc2-ef38a40170b4 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:17.159767 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 881/1754] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:17.193269 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 874/1755] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:17.197159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34232a1a-5e8d-4c31-9bc2-ef38a40170b4 service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:17.236305 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34232a1a-5e8d-4c31-9bc2-ef38a40170b4 service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:17.253638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b17b8d-e9fc-4360-ad3d-70f95267b326 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:17.254178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b17b8d-e9fc-4360-ad3d-70f95267b326 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:17.255118 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b17b8d-e9fc-4360-ad3d-70f95267b326 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:17.259040 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 875/1756] 10.222.0.17 () {54 vars in 961 bytes} [Tue Jul 26 05:00:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 05:00:17.271333 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c45849-dcc9-44d5-9c4d-a4ffda6cb7fb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:17.272004 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c45849-dcc9-44d5-9c4d-a4ffda6cb7fb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:17.272532 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c45849-dcc9-44d5-9c4d-a4ffda6cb7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:17.304058 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d4932-bc37-4816-b846-d23b45e49cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:17.309394 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d4932-bc37-4816-b846-d23b45e49cdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:17.309925 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d4932-bc37-4816-b846-d23b45e49cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:17.310432 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d4932-bc37-4816-b846-d23b45e49cdc admin admin] PATH_INFO: `/v3/projects/1baef349b6654ebd9349c36769519192/users/15eddeafa63e4feca20246f7eb9712fc/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:17.406866 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-29c45849-dcc9-44d5-9c4d-a4ffda6cb7fb None None] MFA Rules not processed for user `5049dba7ffde47b8af3e6fd55b5703e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:17.408904 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-29c45849-dcc9-44d5-9c4d-a4ffda6cb7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:17.416364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d4932-bc37-4816-b846-d23b45e49cdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1baef349b6654ebd9349c36769519192, user_id=15eddeafa63e4feca20246f7eb9712fc, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:17.426076 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d4932-bc37-4816-b846-d23b45e49cdc admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:17.469097 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 882/1757] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:17 2022] PUT /identity/v3/projects/1baef349b6654ebd9349c36769519192/users/15eddeafa63e4feca20246f7eb9712fc/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:17.482825 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb44e8-193f-4afa-8d30-a481ae0a2adc None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:17.484831 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb44e8-193f-4afa-8d30-a481ae0a2adc None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:17.485872 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb44e8-193f-4afa-8d30-a481ae0a2adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:17.512083 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:17.605938 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-1cbb44e8-193f-4afa-8d30-a481ae0a2adc None None] MFA Rules not processed for user `15eddeafa63e4feca20246f7eb9712fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:17.605938 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-1cbb44e8-193f-4afa-8d30-a481ae0a2adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:17.673957 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 876/1758] 10.222.0.17 () {58 vars in 1058 bytes} [Tue Jul 26 05:00:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:00:17.677754 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c45849-dcc9-44d5-9c4d-a4ffda6cb7fb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:17.696946 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c45849-dcc9-44d5-9c4d-a4ffda6cb7fb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:17.783965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 883/1759] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:17.785001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb44e8-193f-4afa-8d30-a481ae0a2adc None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:17.863997 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:17.923586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c09c2-b57a-4758-8fc0-3e05183421e0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5049dba7ffde47b8af3e6fd55b5703e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:17.926913 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c09c2-b57a-4758-8fc0-3e05183421e0 service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:17.927930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c09c2-b57a-4758-8fc0-3e05183421e0 service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:17.928399 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c09c2-b57a-4758-8fc0-3e05183421e0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:17.938513 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a21c09c2-b57a-4758-8fc0-3e05183421e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:17.949825 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb44e8-193f-4afa-8d30-a481ae0a2adc None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:17.971089 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c09c2-b57a-4758-8fc0-3e05183421e0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:17.973459 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c09c2-b57a-4758-8fc0-3e05183421e0 service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:17.973702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d3ea0-35b9-4366-98fd-a0a1948b3cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:17.974194 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d3ea0-35b9-4366-98fd-a0a1948b3cfe admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:17.974472 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d3ea0-35b9-4366-98fd-a0a1948b3cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:17.974753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d3ea0-35b9-4366-98fd-a0a1948b3cfe admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:17.975669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d3ea0-35b9-4366-98fd-a0a1948b3cfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:17.981940 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d3ea0-35b9-4366-98fd-a0a1948b3cfe admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:18.007808 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 877/1760] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:00:18.070173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 884/1761] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:00:17 2022] POST /identity/v3/projects => generated 387 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:18.074001 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c09c2-b57a-4758-8fc0-3e05183421e0 service placement] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:18.094410 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c09c2-b57a-4758-8fc0-3e05183421e0 service placement] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:18.234243 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5eedc-d28a-40fa-ac08-ae8d5d0cda7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:18.234944 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5eedc-d28a-40fa-ac08-ae8d5d0cda7e admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:18.235229 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5eedc-d28a-40fa-ac08-ae8d5d0cda7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:18.235574 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5eedc-d28a-40fa-ac08-ae8d5d0cda7e admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:18.236304 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce5eedc-d28a-40fa-ac08-ae8d5d0cda7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:18.238316 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce5eedc-d28a-40fa-ac08-ae8d5d0cda7e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:18.292395 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d3ea0-35b9-4366-98fd-a0a1948b3cfe admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:18.305391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 878/1762] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:00:18 2022] POST /identity/v3/users => generated 300 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:18.319462 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d3ea0-35b9-4366-98fd-a0a1948b3cfe admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:18.352585 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec8250-3f82-4e78-9dd6-9a20e2cfa70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:18.353065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec8250-3f82-4e78-9dd6-9a20e2cfa70c admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:18.353344 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec8250-3f82-4e78-9dd6-9a20e2cfa70c admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:18.353630 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec8250-3f82-4e78-9dd6-9a20e2cfa70c admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:18.354343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec8250-3f82-4e78-9dd6-9a20e2cfa70c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:18.360560 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec8250-3f82-4e78-9dd6-9a20e2cfa70c admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:18.399343 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5eedc-d28a-40fa-ac08-ae8d5d0cda7e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:18.411369 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 885/1763] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:18.414112 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5eedc-d28a-40fa-ac08-ae8d5d0cda7e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:18.453617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701e145-f4b8-4ec6-9589-c62aecc36b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:18.455638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701e145-f4b8-4ec6-9589-c62aecc36b64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:18.456169 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701e145-f4b8-4ec6-9589-c62aecc36b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:18.456286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701e145-f4b8-4ec6-9589-c62aecc36b64 admin admin] PATH_INFO: `/v3/projects/4e4e4cd47e784a81a599cae89c51caaa/users/53a70dda6772429ca3f1c0a0a2ac67b2/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:18.501972 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701e145-f4b8-4ec6-9589-c62aecc36b64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e4e4cd47e784a81a599cae89c51caaa, user_id=53a70dda6772429ca3f1c0a0a2ac67b2, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:18.506329 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701e145-f4b8-4ec6-9589-c62aecc36b64 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:18.526049 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 879/1764] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:18 2022] PUT /identity/v3/projects/4e4e4cd47e784a81a599cae89c51caaa/users/53a70dda6772429ca3f1c0a0a2ac67b2/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:18.531427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec8250-3f82-4e78-9dd6-9a20e2cfa70c admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:18.564974 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec8250-3f82-4e78-9dd6-9a20e2cfa70c admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:18.771279 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72801e-76c4-4b99-81c3-749410f820e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:18.772661 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72801e-76c4-4b99-81c3-749410f820e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:18.773052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72801e-76c4-4b99-81c3-749410f820e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:18.773429 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72801e-76c4-4b99-81c3-749410f820e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:18.775004 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb72801e-76c4-4b99-81c3-749410f820e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:18.776574 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb72801e-76c4-4b99-81c3-749410f820e8 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:18.798987 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 886/1765] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:18.803412 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701e145-f4b8-4ec6-9589-c62aecc36b64 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:18.819928 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701e145-f4b8-4ec6-9589-c62aecc36b64 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:18.858637 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b5262-0c73-40b9-a9d4-ebd04e2839bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:18.859460 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b5262-0c73-40b9-a9d4-ebd04e2839bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:18.859866 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b5262-0c73-40b9-a9d4-ebd04e2839bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:18.860412 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b5262-0c73-40b9-a9d4-ebd04e2839bb admin admin] PATH_INFO: `/v3/projects/4e4e4cd47e784a81a599cae89c51caaa/users/53a70dda6772429ca3f1c0a0a2ac67b2/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:18.866727 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b5262-0c73-40b9-a9d4-ebd04e2839bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e4e4cd47e784a81a599cae89c51caaa, user_id=53a70dda6772429ca3f1c0a0a2ac67b2, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:18.876435 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b5262-0c73-40b9-a9d4-ebd04e2839bb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:18.942658 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 880/1766] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:18 2022] PUT /identity/v3/projects/4e4e4cd47e784a81a599cae89c51caaa/users/53a70dda6772429ca3f1c0a0a2ac67b2/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:18.955859 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ab935-7b7c-4c95-a2e0-91c9f0d6d0bb None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:18.957194 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ab935-7b7c-4c95-a2e0-91c9f0d6d0bb None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:18.957593 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ab935-7b7c-4c95-a2e0-91c9f0d6d0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:18.967954 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:19.029072 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-3d7ab935-7b7c-4c95-a2e0-91c9f0d6d0bb None None] MFA Rules not processed for user `53a70dda6772429ca3f1c0a0a2ac67b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:19.039368 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-3d7ab935-7b7c-4c95-a2e0-91c9f0d6d0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:19.182703 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 887/1767] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:18 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:19.199263 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b5262-0c73-40b9-a9d4-ebd04e2839bb admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:19.203964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-709b5262-0c73-40b9-a9d4-ebd04e2839bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:19.304227 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b5262-0c73-40b9-a9d4-ebd04e2839bb admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:19.309746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4b453-a917-46a5-b984-ccf0ab1c9aaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:19.310802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4b453-a917-46a5-b984-ccf0ab1c9aaa service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:19.311260 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4b453-a917-46a5-b984-ccf0ab1c9aaa service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:19.311905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4b453-a917-46a5-b984-ccf0ab1c9aaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:19.331519 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad4b453-a917-46a5-b984-ccf0ab1c9aaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:19.338257 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad4b453-a917-46a5-b984-ccf0ab1c9aaa service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:19.355754 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcac65b-9f92-4b27-9b0a-bfb7b4e06ec9 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:19.356265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcac65b-9f92-4b27-9b0a-bfb7b4e06ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:19.356375 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcac65b-9f92-4b27-9b0a-bfb7b4e06ec9 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:19.358051 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 888/1769] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jul 26 05:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 05:00:19.361355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 881/1769] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:19 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:19.438198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf83e9-d634-4632-848d-9c7f3940a602 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:19.438660 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf83e9-d634-4632-848d-9c7f3940a602 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:19.439094 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf83e9-d634-4632-848d-9c7f3940a602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:19.481610 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a5010-4bab-473b-a752-0c018ce5da71 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:19.481987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a5010-4bab-473b-a752-0c018ce5da71 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:19.482266 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a5010-4bab-473b-a752-0c018ce5da71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:19.494366 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-65bf83e9-d634-4632-848d-9c7f3940a602 None None] MFA Rules not processed for user `5049dba7ffde47b8af3e6fd55b5703e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:19.496939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-65bf83e9-d634-4632-848d-9c7f3940a602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:19.519531 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-333a5010-4bab-473b-a752-0c018ce5da71 None None] MFA Rules not processed for user `39de5897bd2d4e2b91c05d8dda5827be`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:19.523055 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-333a5010-4bab-473b-a752-0c018ce5da71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:19.647417 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:19.751128 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 889/1770] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jul 26 05:00:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:19.776841 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 882/1771] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:00:19.806309 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf83e9-d634-4632-848d-9c7f3940a602 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:19.807834 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-65bf83e9-d634-4632-848d-9c7f3940a602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:19.835725 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf83e9-d634-4632-848d-9c7f3940a602 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:19.865315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbc195-cc56-427a-90bd-2e8660c352da service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5049dba7ffde47b8af3e6fd55b5703e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:19.865981 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbc195-cc56-427a-90bd-2e8660c352da service placement] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:19.866267 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbc195-cc56-427a-90bd-2e8660c352da service placement] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:19.866542 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbc195-cc56-427a-90bd-2e8660c352da service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:19.892399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdbc195-cc56-427a-90bd-2e8660c352da service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:19.897969 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdbc195-cc56-427a-90bd-2e8660c352da service placement] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:19.934784 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 890/1772] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:20.217867 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c39b73-9783-4bde-84a4-5cb12412e4f0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:20.218358 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c39b73-9783-4bde-84a4-5cb12412e4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:20.218479 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c39b73-9783-4bde-84a4-5cb12412e4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:20.345617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b9c39b73-9783-4bde-84a4-5cb12412e4f0 None None] MFA Rules not processed for user `f4e86ac513aa4feda193e134c0bc9e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:20.347286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b9c39b73-9783-4bde-84a4-5cb12412e4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:20.648426 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 883/1773] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 432 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:20.671730 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0561c-fce7-47d9-a0d7-7b101ef0c463 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:20.672369 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0561c-fce7-47d9-a0d7-7b101ef0c463 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:20.672369 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0561c-fce7-47d9-a0d7-7b101ef0c463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:20.731775 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-7cc0561c-fce7-47d9-a0d7-7b101ef0c463 None None] MFA Rules not processed for user `f4e86ac513aa4feda193e134c0bc9e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:20.743131 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-7cc0561c-fce7-47d9-a0d7-7b101ef0c463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:20.776914 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 891/1774] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:20.980712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c39b73-9783-4bde-84a4-5cb12412e4f0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:20.982654 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b9c39b73-9783-4bde-84a4-5cb12412e4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:21.072076 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc0561c-fce7-47d9-a0d7-7b101ef0c463 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:21.075143 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-7cc0561c-fce7-47d9-a0d7-7b101ef0c463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:21.254617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c39b73-9783-4bde-84a4-5cb12412e4f0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:21.281768 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f137128-cae9-4a4b-8047-fa4745b6ce3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:21.282731 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f137128-cae9-4a4b-8047-fa4745b6ce3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:21.283076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f137128-cae9-4a4b-8047-fa4745b6ce3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:21.284053 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f137128-cae9-4a4b-8047-fa4745b6ce3f admin admin] PATH_INFO: `/v3/users/13be1d70d27b4be3927b6ee4ee213c4f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:21.290073 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f137128-cae9-4a4b-8047-fa4745b6ce3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=13be1d70d27b4be3927b6ee4ee213c4f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:21.297181 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f137128-cae9-4a4b-8047-fa4745b6ce3f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:21.367097 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:21.413051 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc0561c-fce7-47d9-a0d7-7b101ef0c463 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:21.429695 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc9885-a85b-48fa-849d-7a9e4fc680c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:21.430367 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc9885-a85b-48fa-849d-7a9e4fc680c7 service nova] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:21.430655 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc9885-a85b-48fa-849d-7a9e4fc680c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:21.430933 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc9885-a85b-48fa-849d-7a9e4fc680c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:21.448731 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc9885-a85b-48fa-849d-7a9e4fc680c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:21.452659 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3f137128-cae9-4a4b-8047-fa4745b6ce3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13be1d70d27b4be3927b6ee4ee213c4f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:21.452881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc9885-a85b-48fa-849d-7a9e4fc680c7 service nova] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:21.467719 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3f137128-cae9-4a4b-8047-fa4745b6ce3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13be1d70d27b4be3927b6ee4ee213c4f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:21.469744 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3f137128-cae9-4a4b-8047-fa4745b6ce3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13be1d70d27b4be3927b6ee4ee213c4f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:21.478696 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 892/1775] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:21.480023 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc9885-a85b-48fa-849d-7a9e4fc680c7 service nova] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:21.482514 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-33cc9885-a85b-48fa-849d-7a9e4fc680c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:21.491119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3f137128-cae9-4a4b-8047-fa4745b6ce3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13be1d70d27b4be3927b6ee4ee213c4f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:21.491921 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3f137128-cae9-4a4b-8047-fa4745b6ce3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13be1d70d27b4be3927b6ee4ee213c4f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:21.498844 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:21.521138 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc9885-a85b-48fa-849d-7a9e4fc680c7 service nova] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:21.533492 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131541de-dfc2-4bf6-8001-65169ff48bc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:21.536924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 884/1776] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:00:20 2022] DELETE /identity/v3/users/13be1d70d27b4be3927b6ee4ee213c4f => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:21.537387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131541de-dfc2-4bf6-8001-65169ff48bc7 service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:21.537754 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131541de-dfc2-4bf6-8001-65169ff48bc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:21.538188 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131541de-dfc2-4bf6-8001-65169ff48bc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:21.541384 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1896e-f6a3-46c0-977a-c8cf96342f4a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:21.541698 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1896e-f6a3-46c0-977a-c8cf96342f4a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:21.543788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-131541de-dfc2-4bf6-8001-65169ff48bc7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:21.545899 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1896e-f6a3-46c0-977a-c8cf96342f4a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:21.547607 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 885/1777] 10.222.0.17 () {54 vars in 912 bytes} [Tue Jul 26 05:00:21 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 05:00:21.554664 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19f997-e2d7-442d-b923-05168ab1813a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:21.556991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19f997-e2d7-442d-b923-05168ab1813a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:21.557675 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19f997-e2d7-442d-b923-05168ab1813a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:21.605956 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8d19f997-e2d7-442d-b923-05168ab1813a None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:21.610717 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8d19f997-e2d7-442d-b923-05168ab1813a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:21.723536 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131541de-dfc2-4bf6-8001-65169ff48bc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:21.729834 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131541de-dfc2-4bf6-8001-65169ff48bc7 service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:21.756311 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 893/1778] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:21.758017 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68726119-9244-4df9-89cf-474e34dea925 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:21.758310 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68726119-9244-4df9-89cf-474e34dea925 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:21.758957 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68726119-9244-4df9-89cf-474e34dea925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:21.792484 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 886/1779] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:21.824894 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-68726119-9244-4df9-89cf-474e34dea925 None None] MFA Rules not processed for user `dbdb8997891241e9b2e829feb8023f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:21.826285 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-68726119-9244-4df9-89cf-474e34dea925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:21.901778 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 894/1780] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jul 26 05:00:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:21.925315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19f997-e2d7-442d-b923-05168ab1813a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:21.927207 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8d19f997-e2d7-442d-b923-05168ab1813a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:21.948783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19f997-e2d7-442d-b923-05168ab1813a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:21.963778 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5f795-501e-487b-bd72-efa445075778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:21.964805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5f795-501e-487b-bd72-efa445075778 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:21.965227 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5f795-501e-487b-bd72-efa445075778 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:21.965552 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5f795-501e-487b-bd72-efa445075778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:21.999131 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5f795-501e-487b-bd72-efa445075778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:22.002598 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5f795-501e-487b-bd72-efa445075778 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:22.038345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 887/1781] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:22.182360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68726119-9244-4df9-89cf-474e34dea925 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:22.184704 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-68726119-9244-4df9-89cf-474e34dea925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:22.211293 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68726119-9244-4df9-89cf-474e34dea925 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:22.233565 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a0150-6388-4ea7-9742-3c4e7c540cf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:22.234271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a0150-6388-4ea7-9742-3c4e7c540cf2 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:22.234560 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a0150-6388-4ea7-9742-3c4e7c540cf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:22.234843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a0150-6388-4ea7-9742-3c4e7c540cf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:22.268951 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707a0150-6388-4ea7-9742-3c4e7c540cf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:22.270570 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707a0150-6388-4ea7-9742-3c4e7c540cf2 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:22.305907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 895/1782] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:22.601612 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1615628-bb02-4bc0-9b34-9dabdac891d3 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:22.602132 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1615628-bb02-4bc0-9b34-9dabdac891d3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:22.602226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1615628-bb02-4bc0-9b34-9dabdac891d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:22.734191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f1615628-bb02-4bc0-9b34-9dabdac891d3 None None] MFA Rules not processed for user `3a8f3eb811ae40ad9cb64261f95a992e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:22.735486 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f1615628-bb02-4bc0-9b34-9dabdac891d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:22.893026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a0150-6388-4ea7-9742-3c4e7c540cf2 service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:22.896484 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-707a0150-6388-4ea7-9742-3c4e7c540cf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:22.937116 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a0150-6388-4ea7-9742-3c4e7c540cf2 service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:22.956496 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264b272-0bea-4eaa-8992-6b304ffdc519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:22.958139 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 888/1783] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jul 26 05:00:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:22.962088 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264b272-0bea-4eaa-8992-6b304ffdc519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:22.963611 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264b272-0bea-4eaa-8992-6b304ffdc519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:22.964823 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264b272-0bea-4eaa-8992-6b304ffdc519 admin admin] PATH_INFO: `/v3/users/d9e53fc3e5e345f0b9bc6efa3745441f` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:22.972806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264b272-0bea-4eaa-8992-6b304ffdc519 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9e53fc3e5e345f0b9bc6efa3745441f)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:22.976653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264b272-0bea-4eaa-8992-6b304ffdc519 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:23.112406 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f264b272-0bea-4eaa-8992-6b304ffdc519 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9e53fc3e5e345f0b9bc6efa3745441f'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:23.123644 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f264b272-0bea-4eaa-8992-6b304ffdc519 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9e53fc3e5e345f0b9bc6efa3745441f'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:23.137136 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f264b272-0bea-4eaa-8992-6b304ffdc519 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9e53fc3e5e345f0b9bc6efa3745441f'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:23.138717 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f264b272-0bea-4eaa-8992-6b304ffdc519 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9e53fc3e5e345f0b9bc6efa3745441f'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:23.139755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f264b272-0bea-4eaa-8992-6b304ffdc519 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9e53fc3e5e345f0b9bc6efa3745441f'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:23.161937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:23.178684 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 896/1784] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:00:22 2022] DELETE /identity/v3/users/d9e53fc3e5e345f0b9bc6efa3745441f => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:23.180358 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1615628-bb02-4bc0-9b34-9dabdac891d3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:23.182077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f1615628-bb02-4bc0-9b34-9dabdac891d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:23.210184 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1615628-bb02-4bc0-9b34-9dabdac891d3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:23.286769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264b272-0bea-4eaa-8992-6b304ffdc519 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:23.288300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-f264b272-0bea-4eaa-8992-6b304ffdc519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:23.388796 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:23.493072 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72c1cd-0acc-4d1d-b69b-8cbd0335154c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:23.498452 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72c1cd-0acc-4d1d-b69b-8cbd0335154c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:23.499660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72c1cd-0acc-4d1d-b69b-8cbd0335154c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:23.502690 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72c1cd-0acc-4d1d-b69b-8cbd0335154c admin admin] PATH_INFO: `/v3/projects/fc0d4aa379034eb7839167e0ee119be3` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:23.517894 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72c1cd-0acc-4d1d-b69b-8cbd0335154c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc0d4aa379034eb7839167e0ee119be3)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:23.527996 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72c1cd-0acc-4d1d-b69b-8cbd0335154c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:23.662173 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fe72c1cd-0acc-4d1d-b69b-8cbd0335154c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc0d4aa379034eb7839167e0ee119be3', 'request_id': 'req-fe72c1cd-0acc-4d1d-b69b-8cbd0335154c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:23.670167 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264b272-0bea-4eaa-8992-6b304ffdc519 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:23.688625 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fe72c1cd-0acc-4d1d-b69b-8cbd0335154c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc0d4aa379034eb7839167e0ee119be3', 'request_id': 'req-fe72c1cd-0acc-4d1d-b69b-8cbd0335154c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:23.706245 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fe72c1cd-0acc-4d1d-b69b-8cbd0335154c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc0d4aa379034eb7839167e0ee119be3', 'request_id': 'req-fe72c1cd-0acc-4d1d-b69b-8cbd0335154c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:23.755953 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fe72c1cd-0acc-4d1d-b69b-8cbd0335154c admin admin] The token cache is being invalidate because project fc0d4aa379034eb7839167e0ee119be3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:00:23.756259 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fe72c1cd-0acc-4d1d-b69b-8cbd0335154c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:23.764168 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fe72c1cd-0acc-4d1d-b69b-8cbd0335154c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:23.771989 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 889/1785] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:00:23 2022] DELETE /identity/v3/projects/fc0d4aa379034eb7839167e0ee119be3 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:23.773220 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72c1cd-0acc-4d1d-b69b-8cbd0335154c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:23.776188 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fe72c1cd-0acc-4d1d-b69b-8cbd0335154c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:23.882194 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095adf8-cf74-44ae-9114-8665a8f34113 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5218319aaa5424882714bffcbeee730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:23.884490 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095adf8-cf74-44ae-9114-8665a8f34113 service swift] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:23.885200 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095adf8-cf74-44ae-9114-8665a8f34113 service swift] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:23.885957 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095adf8-cf74-44ae-9114-8665a8f34113 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:23.892949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a095adf8-cf74-44ae-9114-8665a8f34113 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:24.031400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72c1cd-0acc-4d1d-b69b-8cbd0335154c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:24.060649 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202a135-e388-4eae-b7d6-1f2a967f7c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:24.066369 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202a135-e388-4eae-b7d6-1f2a967f7c25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:24.066696 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202a135-e388-4eae-b7d6-1f2a967f7c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:24.067055 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202a135-e388-4eae-b7d6-1f2a967f7c25 admin admin] PATH_INFO: `/v3/users/c1d1cb4c636b4445ab2d2fec500c6c53` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:24.094437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202a135-e388-4eae-b7d6-1f2a967f7c25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1d1cb4c636b4445ab2d2fec500c6c53)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:24.100084 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202a135-e388-4eae-b7d6-1f2a967f7c25 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:24.163493 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095adf8-cf74-44ae-9114-8665a8f34113 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:24.169142 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095adf8-cf74-44ae-9114-8665a8f34113 service swift] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:24.214242 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 897/1786] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jul 26 05:00:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 928 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 05:00:24.248056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5202a135-e388-4eae-b7d6-1f2a967f7c25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1d1cb4c636b4445ab2d2fec500c6c53'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:24.274040 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5202a135-e388-4eae-b7d6-1f2a967f7c25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1d1cb4c636b4445ab2d2fec500c6c53'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:24.281439 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5202a135-e388-4eae-b7d6-1f2a967f7c25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1d1cb4c636b4445ab2d2fec500c6c53'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:24.305238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5202a135-e388-4eae-b7d6-1f2a967f7c25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1d1cb4c636b4445ab2d2fec500c6c53'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:24.309232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5202a135-e388-4eae-b7d6-1f2a967f7c25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1d1cb4c636b4445ab2d2fec500c6c53'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:24.368159 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:24.379912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 890/1787] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:00:23 2022] DELETE /identity/v3/users/c1d1cb4c636b4445ab2d2fec500c6c53 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:24.768121 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095adf8-cf74-44ae-9114-8665a8f34113 service swift] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:24.769751 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a095adf8-cf74-44ae-9114-8665a8f34113 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:24.840212 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095adf8-cf74-44ae-9114-8665a8f34113 service swift] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:24.906104 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:25.163725 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384565b-1675-43cc-8eb7-68fd0a424d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:25.164739 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384565b-1675-43cc-8eb7-68fd0a424d22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:25.165160 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384565b-1675-43cc-8eb7-68fd0a424d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:25.165472 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384565b-1675-43cc-8eb7-68fd0a424d22 admin admin] PATH_INFO: `/v3/projects/4e1d08f366f24923bb7b43a52178d51e` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:25.179160 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384565b-1675-43cc-8eb7-68fd0a424d22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e1d08f366f24923bb7b43a52178d51e)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:25.185500 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384565b-1675-43cc-8eb7-68fd0a424d22 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:25.219482 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693ea37-0ad8-466b-b086-9ec9d5ab0053 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:25.220867 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693ea37-0ad8-466b-b086-9ec9d5ab0053 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:25.221303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693ea37-0ad8-466b-b086-9ec9d5ab0053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:25.342240 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e693ea37-0ad8-466b-b086-9ec9d5ab0053 None None] MFA Rules not processed for user `f0cc1af7d7d642889a1a11103f087e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:25.345607 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e693ea37-0ad8-466b-b086-9ec9d5ab0053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:25.500969 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6384565b-1675-43cc-8eb7-68fd0a424d22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e1d08f366f24923bb7b43a52178d51e', 'request_id': 'req-6384565b-1675-43cc-8eb7-68fd0a424d22'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:25.535930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6384565b-1675-43cc-8eb7-68fd0a424d22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e1d08f366f24923bb7b43a52178d51e', 'request_id': 'req-6384565b-1675-43cc-8eb7-68fd0a424d22'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:25.562965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6384565b-1675-43cc-8eb7-68fd0a424d22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e1d08f366f24923bb7b43a52178d51e', 'request_id': 'req-6384565b-1675-43cc-8eb7-68fd0a424d22'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:25.610213 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6384565b-1675-43cc-8eb7-68fd0a424d22 admin admin] The token cache is being invalidate because project 4e1d08f366f24923bb7b43a52178d51e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:00:25.613459 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6384565b-1675-43cc-8eb7-68fd0a424d22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:25.620449 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6384565b-1675-43cc-8eb7-68fd0a424d22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:25.633415 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 898/1788] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:00:24 2022] DELETE /identity/v3/projects/4e1d08f366f24923bb7b43a52178d51e => generated 0 bytes in 866 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:25.639490 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384565b-1675-43cc-8eb7-68fd0a424d22 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:25.641663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-6384565b-1675-43cc-8eb7-68fd0a424d22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:25.895436 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 891/1789] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 677 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:00:25.896669 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693ea37-0ad8-466b-b086-9ec9d5ab0053 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:25.900499 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e693ea37-0ad8-466b-b086-9ec9d5ab0053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:25.949215 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693ea37-0ad8-466b-b086-9ec9d5ab0053 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:26.114472 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384565b-1675-43cc-8eb7-68fd0a424d22 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:26.159099 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85bf061-5c82-4d97-8d1e-0430537cd931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:26.161901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bf061-5c82-4d97-8d1e-0430537cd931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:26.171824 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bf061-5c82-4d97-8d1e-0430537cd931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:26.173045 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bf061-5c82-4d97-8d1e-0430537cd931 admin admin] PATH_INFO: `/v3/users/7acc033a5bcc4cabb12bf180e7a31f89` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:26.184437 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85bf061-5c82-4d97-8d1e-0430537cd931 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7acc033a5bcc4cabb12bf180e7a31f89)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:26.200298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85bf061-5c82-4d97-8d1e-0430537cd931 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:26.220394 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a5fbb-a23a-42ba-ba35-59e39e0da28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:26.221283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a5fbb-a23a-42ba-ba35-59e39e0da28c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:26.221730 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a5fbb-a23a-42ba-ba35-59e39e0da28c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:26.221730 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a5fbb-a23a-42ba-ba35-59e39e0da28c admin admin] PATH_INFO: `/v3/projects/5de03239c4e24cc7bf6dec1cd2da2567` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:26.236541 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a5fbb-a23a-42ba-ba35-59e39e0da28c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5de03239c4e24cc7bf6dec1cd2da2567)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:26.238667 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a5fbb-a23a-42ba-ba35-59e39e0da28c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:26.539646 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c85bf061-5c82-4d97-8d1e-0430537cd931 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7acc033a5bcc4cabb12bf180e7a31f89'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:26.555718 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-481a5fbb-a23a-42ba-ba35-59e39e0da28c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5de03239c4e24cc7bf6dec1cd2da2567', 'request_id': 'req-481a5fbb-a23a-42ba-ba35-59e39e0da28c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:26.566166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:26.577719 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c85bf061-5c82-4d97-8d1e-0430537cd931 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7acc033a5bcc4cabb12bf180e7a31f89'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:26.583750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-481a5fbb-a23a-42ba-ba35-59e39e0da28c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5de03239c4e24cc7bf6dec1cd2da2567', 'request_id': 'req-481a5fbb-a23a-42ba-ba35-59e39e0da28c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:26.615904 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-481a5fbb-a23a-42ba-ba35-59e39e0da28c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5de03239c4e24cc7bf6dec1cd2da2567', 'request_id': 'req-481a5fbb-a23a-42ba-ba35-59e39e0da28c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:26.616482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c85bf061-5c82-4d97-8d1e-0430537cd931 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7acc033a5bcc4cabb12bf180e7a31f89'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:26.617993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c85bf061-5c82-4d97-8d1e-0430537cd931 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7acc033a5bcc4cabb12bf180e7a31f89'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:26.629200 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c85bf061-5c82-4d97-8d1e-0430537cd931 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7acc033a5bcc4cabb12bf180e7a31f89'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:26.643301 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-481a5fbb-a23a-42ba-ba35-59e39e0da28c admin admin] The token cache is being invalidate because project 5de03239c4e24cc7bf6dec1cd2da2567 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:00:26.643663 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-481a5fbb-a23a-42ba-ba35-59e39e0da28c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:26.651878 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-481a5fbb-a23a-42ba-ba35-59e39e0da28c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:26.654802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 892/1790] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:00:25 2022] DELETE /identity/v3/projects/5de03239c4e24cc7bf6dec1cd2da2567 => generated 0 bytes in 759 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:26.656628 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace71ab-6ca7-4a4b-8f6e-8e0a4316433d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:26.657175 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace71ab-6ca7-4a4b-8f6e-8e0a4316433d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:26.658443 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace71ab-6ca7-4a4b-8f6e-8e0a4316433d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:26.687776 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 899/1791] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:00:25 2022] DELETE /identity/v3/users/7acc033a5bcc4cabb12bf180e7a31f89 => generated 0 bytes in 1049 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:00:26.688994 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85bf061-5c82-4d97-8d1e-0430537cd931 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:26.692127 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c85bf061-5c82-4d97-8d1e-0430537cd931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:26.723405 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1ace71ab-6ca7-4a4b-8f6e-8e0a4316433d None None] MFA Rules not processed for user `f0cc1af7d7d642889a1a11103f087e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:26.724976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1ace71ab-6ca7-4a4b-8f6e-8e0a4316433d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:26.743274 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:26.833781 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 893/1792] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:26.843423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9beeb9c-de57-4387-96d5-1a09ecb1914e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:26.845277 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9beeb9c-de57-4387-96d5-1a09ecb1914e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:26.846316 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9beeb9c-de57-4387-96d5-1a09ecb1914e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:26.880793 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85bf061-5c82-4d97-8d1e-0430537cd931 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:26.898041 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f625332-2a4e-4445-9aba-c26e573cf0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:26.898783 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f625332-2a4e-4445-9aba-c26e573cf0f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:26.899084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f625332-2a4e-4445-9aba-c26e573cf0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:26.899366 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f625332-2a4e-4445-9aba-c26e573cf0f9 admin admin] PATH_INFO: `/v3/projects/35463c490516487c936b93bcd5407ba6` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:26.907654 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f625332-2a4e-4445-9aba-c26e573cf0f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35463c490516487c936b93bcd5407ba6)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:26.915058 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f625332-2a4e-4445-9aba-c26e573cf0f9 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:26.917965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f9beeb9c-de57-4387-96d5-1a09ecb1914e None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:26.925330 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f9beeb9c-de57-4387-96d5-1a09ecb1914e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:26.946826 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 894/1793] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:26.950054 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9beeb9c-de57-4387-96d5-1a09ecb1914e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:26.955606 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f9beeb9c-de57-4387-96d5-1a09ecb1914e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:27.043966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8f625332-2a4e-4445-9aba-c26e573cf0f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35463c490516487c936b93bcd5407ba6', 'request_id': 'req-8f625332-2a4e-4445-9aba-c26e573cf0f9'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:27.059521 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8f625332-2a4e-4445-9aba-c26e573cf0f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35463c490516487c936b93bcd5407ba6', 'request_id': 'req-8f625332-2a4e-4445-9aba-c26e573cf0f9'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:27.095868 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8f625332-2a4e-4445-9aba-c26e573cf0f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35463c490516487c936b93bcd5407ba6', 'request_id': 'req-8f625332-2a4e-4445-9aba-c26e573cf0f9'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:27.106715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9beeb9c-de57-4387-96d5-1a09ecb1914e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:27.121820 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8f625332-2a4e-4445-9aba-c26e573cf0f9 admin admin] The token cache is being invalidate because project 35463c490516487c936b93bcd5407ba6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:00:27.122387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8f625332-2a4e-4445-9aba-c26e573cf0f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:27.125581 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8f625332-2a4e-4445-9aba-c26e573cf0f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:27.129906 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 900/1794] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:00:26 2022] DELETE /identity/v3/projects/35463c490516487c936b93bcd5407ba6 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:27.135336 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49170abf-41cd-49a8-8a07-88062ff15c0a None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:27.136077 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49170abf-41cd-49a8-8a07-88062ff15c0a None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:27.136577 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49170abf-41cd-49a8-8a07-88062ff15c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:27.170045 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-49170abf-41cd-49a8-8a07-88062ff15c0a None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:27.172305 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-49170abf-41cd-49a8-8a07-88062ff15c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:27.295821 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:27.309451 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86e111-5b50-4bff-8700-0c4a75359e85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:27.312289 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86e111-5b50-4bff-8700-0c4a75359e85 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:27.314697 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86e111-5b50-4bff-8700-0c4a75359e85 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:27.315120 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86e111-5b50-4bff-8700-0c4a75359e85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:27.323841 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2d86e111-5b50-4bff-8700-0c4a75359e85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:27.327700 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 901/1795] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:27.329032 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49170abf-41cd-49a8-8a07-88062ff15c0a None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:27.334000 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-49170abf-41cd-49a8-8a07-88062ff15c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:27.366747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49170abf-41cd-49a8-8a07-88062ff15c0a None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:27.385295 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8952a4-6cf5-4c19-a9f2-e64c072a3e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:27.386796 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8952a4-6cf5-4c19-a9f2-e64c072a3e5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:27.387396 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8952a4-6cf5-4c19-a9f2-e64c072a3e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:27.387915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8952a4-6cf5-4c19-a9f2-e64c072a3e5a admin admin] PATH_INFO: `/v3/users/8736ccfd969f41aa8adf822c2bb32910` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:27.391986 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8952a4-6cf5-4c19-a9f2-e64c072a3e5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8736ccfd969f41aa8adf822c2bb32910)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:27.397531 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8952a4-6cf5-4c19-a9f2-e64c072a3e5a admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:27.474737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:27.519239 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86e111-5b50-4bff-8700-0c4a75359e85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:27.526759 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86e111-5b50-4bff-8700-0c4a75359e85 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:27.558170 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8e8952a4-6cf5-4c19-a9f2-e64c072a3e5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8736ccfd969f41aa8adf822c2bb32910'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:27.561225 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 895/1796] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:26 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:27.563917 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86e111-5b50-4bff-8700-0c4a75359e85 service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:27.579879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8e8952a4-6cf5-4c19-a9f2-e64c072a3e5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8736ccfd969f41aa8adf822c2bb32910'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:27.594433 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86e111-5b50-4bff-8700-0c4a75359e85 service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:27.600114 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8e8952a4-6cf5-4c19-a9f2-e64c072a3e5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8736ccfd969f41aa8adf822c2bb32910'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:27.601654 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8e8952a4-6cf5-4c19-a9f2-e64c072a3e5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8736ccfd969f41aa8adf822c2bb32910'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:27.603232 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8e8952a4-6cf5-4c19-a9f2-e64c072a3e5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8736ccfd969f41aa8adf822c2bb32910'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:27.627270 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0835b2e-2cf2-4eb7-a8fa-871282d0b96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:27.629204 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0835b2e-2cf2-4eb7-a8fa-871282d0b96c admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:27.630030 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0835b2e-2cf2-4eb7-a8fa-871282d0b96c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:27.630587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0835b2e-2cf2-4eb7-a8fa-871282d0b96c admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:27.632589 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0835b2e-2cf2-4eb7-a8fa-871282d0b96c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:00:27.633802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0835b2e-2cf2-4eb7-a8fa-871282d0b96c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:27.640143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0835b2e-2cf2-4eb7-a8fa-871282d0b96c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:27.677349 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 902/1797] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:00:27 2022] DELETE /identity/v3/users/8736ccfd969f41aa8adf822c2bb32910 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:27.677349 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8952a4-6cf5-4c19-a9f2-e64c072a3e5a admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:27.678505 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-8e8952a4-6cf5-4c19-a9f2-e64c072a3e5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:27.683684 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 896/1798] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:00:27.689352 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0835b2e-2cf2-4eb7-a8fa-871282d0b96c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:27.705826 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d0835b2e-2cf2-4eb7-a8fa-871282d0b96c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:27.778737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0835b2e-2cf2-4eb7-a8fa-871282d0b96c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:27.934848 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de2ec4-3e97-4ae2-acd3-ccf7ab96ee4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:27.935769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de2ec4-3e97-4ae2-acd3-ccf7ab96ee4d admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:27.936250 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de2ec4-3e97-4ae2-acd3-ccf7ab96ee4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:27.936654 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de2ec4-3e97-4ae2-acd3-ccf7ab96ee4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:27.937549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6de2ec4-3e97-4ae2-acd3-ccf7ab96ee4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:27.943314 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6de2ec4-3e97-4ae2-acd3-ccf7ab96ee4d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:28.019876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 897/1799] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:00:27 2022] POST /identity/v3/projects => generated 379 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:28.029494 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de2ec4-3e97-4ae2-acd3-ccf7ab96ee4d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:28.098903 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8952a4-6cf5-4c19-a9f2-e64c072a3e5a admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:28.297877 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de2ec4-3e97-4ae2-acd3-ccf7ab96ee4d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:28.311723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647f049-ec3a-4995-a870-ba40e304f02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:28.314270 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647f049-ec3a-4995-a870-ba40e304f02d admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:28.315666 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647f049-ec3a-4995-a870-ba40e304f02d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:28.316026 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647f049-ec3a-4995-a870-ba40e304f02d admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:28.317707 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5647f049-ec3a-4995-a870-ba40e304f02d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:28.322061 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5647f049-ec3a-4995-a870-ba40e304f02d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:28.451689 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 898/1800] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:00:28 2022] POST /identity/v3/users => generated 290 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:28.460359 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647f049-ec3a-4995-a870-ba40e304f02d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:28.478806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213f1a9-8f71-4bbe-aa16-7aeb04d8f57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:28.484867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213f1a9-8f71-4bbe-aa16-7aeb04d8f57b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:28.484867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213f1a9-8f71-4bbe-aa16-7aeb04d8f57b admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:28.485159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213f1a9-8f71-4bbe-aa16-7aeb04d8f57b admin admin] PATH_INFO: `/v3/projects/f50bae84e5294d469fc867a01cdb2602` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:28.489450 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213f1a9-8f71-4bbe-aa16-7aeb04d8f57b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f50bae84e5294d469fc867a01cdb2602)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:28.498988 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213f1a9-8f71-4bbe-aa16-7aeb04d8f57b admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:28.500658 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647f049-ec3a-4995-a870-ba40e304f02d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:28.510472 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080e9dc-d39f-407b-b3dd-6933c86c3826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:28.513369 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080e9dc-d39f-407b-b3dd-6933c86c3826 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:28.515139 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080e9dc-d39f-407b-b3dd-6933c86c3826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:28.518712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080e9dc-d39f-407b-b3dd-6933c86c3826 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:28.520108 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e080e9dc-d39f-407b-b3dd-6933c86c3826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:28.524310 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e080e9dc-d39f-407b-b3dd-6933c86c3826 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:28.584847 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 899/1801] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:28.590917 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080e9dc-d39f-407b-b3dd-6933c86c3826 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:28.636804 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:28.656497 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080e9dc-d39f-407b-b3dd-6933c86c3826 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:28.678951 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122a067-6611-4daf-8409-74b04cea9f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:28.682939 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122a067-6611-4daf-8409-74b04cea9f6d admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:28.684169 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122a067-6611-4daf-8409-74b04cea9f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:28.684833 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122a067-6611-4daf-8409-74b04cea9f6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:28.686593 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8122a067-6611-4daf-8409-74b04cea9f6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:28.692852 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8122a067-6611-4daf-8409-74b04cea9f6d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:28.730873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9213f1a9-8f71-4bbe-aa16-7aeb04d8f57b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f50bae84e5294d469fc867a01cdb2602', 'request_id': 'req-9213f1a9-8f71-4bbe-aa16-7aeb04d8f57b'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:28.743993 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 900/1802] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:28.743993 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122a067-6611-4daf-8409-74b04cea9f6d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:28.746718 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8122a067-6611-4daf-8409-74b04cea9f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:28.767806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9213f1a9-8f71-4bbe-aa16-7aeb04d8f57b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f50bae84e5294d469fc867a01cdb2602', 'request_id': 'req-9213f1a9-8f71-4bbe-aa16-7aeb04d8f57b'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:28.817313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9213f1a9-8f71-4bbe-aa16-7aeb04d8f57b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f50bae84e5294d469fc867a01cdb2602', 'request_id': 'req-9213f1a9-8f71-4bbe-aa16-7aeb04d8f57b'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:28.855671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9213f1a9-8f71-4bbe-aa16-7aeb04d8f57b admin admin] The token cache is being invalidate because project f50bae84e5294d469fc867a01cdb2602 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:00:28.856434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9213f1a9-8f71-4bbe-aa16-7aeb04d8f57b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:28.857609 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9213f1a9-8f71-4bbe-aa16-7aeb04d8f57b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:28.862352 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:28.863077 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 903/1803] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:00:27 2022] DELETE /identity/v3/projects/f50bae84e5294d469fc867a01cdb2602 => generated 0 bytes in 1188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:28.865785 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213f1a9-8f71-4bbe-aa16-7aeb04d8f57b admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:28.870845 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-9213f1a9-8f71-4bbe-aa16-7aeb04d8f57b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:29.038090 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122a067-6611-4daf-8409-74b04cea9f6d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:29.070790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213f1a9-8f71-4bbe-aa16-7aeb04d8f57b admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:29.092864 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c9d0c-3491-4d2f-ba63-a4e0b1246d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:29.094015 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c9d0c-3491-4d2f-ba63-a4e0b1246d30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:29.094433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c9d0c-3491-4d2f-ba63-a4e0b1246d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:29.094846 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c9d0c-3491-4d2f-ba63-a4e0b1246d30 admin admin] PATH_INFO: `/v3/projects/d6a6c2ad05e94d8c9bed191360d80c87/users/69e8fcabe6f74fc2992b7bd94a4310f8/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:29.141596 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219c9d0c-3491-4d2f-ba63-a4e0b1246d30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6a6c2ad05e94d8c9bed191360d80c87, user_id=69e8fcabe6f74fc2992b7bd94a4310f8, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:29.147950 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219c9d0c-3491-4d2f-ba63-a4e0b1246d30 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:29.187921 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 904/1804] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:28 2022] PUT /identity/v3/projects/d6a6c2ad05e94d8c9bed191360d80c87/users/69e8fcabe6f74fc2992b7bd94a4310f8/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:29.192362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e788fc-1cee-49c6-902f-43908f091825 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:29.194095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e788fc-1cee-49c6-902f-43908f091825 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:29.194617 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e788fc-1cee-49c6-902f-43908f091825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:29.260588 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-d4e788fc-1cee-49c6-902f-43908f091825 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:29.263591 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-d4e788fc-1cee-49c6-902f-43908f091825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:29.305553 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060e3ff-36ac-449c-aa26-80525cad057f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:29.306446 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060e3ff-36ac-449c-aa26-80525cad057f service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:29.312729 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060e3ff-36ac-449c-aa26-80525cad057f service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:29.314305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060e3ff-36ac-449c-aa26-80525cad057f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:29.325005 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c060e3ff-36ac-449c-aa26-80525cad057f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:29.502964 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 905/1805] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:29.504488 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd32d56-0fa6-45d1-a621-06e7c53d2796 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:29.504654 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd32d56-0fa6-45d1-a621-06e7c53d2796 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:29.504927 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd32d56-0fa6-45d1-a621-06e7c53d2796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:29.619843 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060e3ff-36ac-449c-aa26-80525cad057f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:29.622522 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060e3ff-36ac-449c-aa26-80525cad057f service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:29.633284 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-6bd32d56-0fa6-45d1-a621-06e7c53d2796 None None] MFA Rules not processed for user `69e8fcabe6f74fc2992b7bd94a4310f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:29.634849 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-6bd32d56-0fa6-45d1-a621-06e7c53d2796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:29.672896 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 901/1806] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 933 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:29.677576 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70535c70-d1a1-4bb7-a48b-f1c0b0355463 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:29.677865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70535c70-d1a1-4bb7-a48b-f1c0b0355463 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:29.678267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70535c70-d1a1-4bb7-a48b-f1c0b0355463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:29.726072 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:29.833522 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-70535c70-d1a1-4bb7-a48b-f1c0b0355463 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:29.835727 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-70535c70-d1a1-4bb7-a48b-f1c0b0355463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:29.878090 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 902/1807] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:29.885429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70535c70-d1a1-4bb7-a48b-f1c0b0355463 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:29.929422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 906/1808] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 425 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:29.930711 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd32d56-0fa6-45d1-a621-06e7c53d2796 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:29.959198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd32d56-0fa6-45d1-a621-06e7c53d2796 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:29.969112 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70535c70-d1a1-4bb7-a48b-f1c0b0355463 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:29.994509 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520110c-5ee8-46ee-a2bf-1a4f64a39202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:29.995359 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520110c-5ee8-46ee-a2bf-1a4f64a39202 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:29.995734 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520110c-5ee8-46ee-a2bf-1a4f64a39202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:29.996003 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520110c-5ee8-46ee-a2bf-1a4f64a39202 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:29.996755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520110c-5ee8-46ee-a2bf-1a4f64a39202 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:30.006413 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978bf146-488a-4491-8553-49e28a64e407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:30.015864 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520110c-5ee8-46ee-a2bf-1a4f64a39202 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:30.016264 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978bf146-488a-4491-8553-49e28a64e407 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:30.016264 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978bf146-488a-4491-8553-49e28a64e407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:30.016264 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978bf146-488a-4491-8553-49e28a64e407 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:30.016264 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978bf146-488a-4491-8553-49e28a64e407 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:00:30.016264 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978bf146-488a-4491-8553-49e28a64e407 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:30.016264 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978bf146-488a-4491-8553-49e28a64e407 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:30.098879 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 903/1809] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:00:30.109388 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978bf146-488a-4491-8553-49e28a64e407 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:30.142263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 907/1810] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:00:29 2022] POST /identity/v3/projects => generated 381 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:30.143490 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520110c-5ee8-46ee-a2bf-1a4f64a39202 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:30.162512 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978bf146-488a-4491-8553-49e28a64e407 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:30.344328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced13aa2-65e1-410c-bb51-80963260a1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:30.345028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced13aa2-65e1-410c-bb51-80963260a1f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:30.345263 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced13aa2-65e1-410c-bb51-80963260a1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:30.345540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced13aa2-65e1-410c-bb51-80963260a1f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:30.346231 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced13aa2-65e1-410c-bb51-80963260a1f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:30.348347 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced13aa2-65e1-410c-bb51-80963260a1f2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:30.463758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 904/1811] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:00:30 2022] POST /identity/v3/projects => generated 371 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:30.467539 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced13aa2-65e1-410c-bb51-80963260a1f2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:30.496812 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:30.529168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520110c-5ee8-46ee-a2bf-1a4f64a39202 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:30.734560 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced13aa2-65e1-410c-bb51-80963260a1f2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:30.748962 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284850cc-e46a-4d6f-8099-82c038c2a163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:30.749992 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284850cc-e46a-4d6f-8099-82c038c2a163 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:30.750556 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284850cc-e46a-4d6f-8099-82c038c2a163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:30.751083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284850cc-e46a-4d6f-8099-82c038c2a163 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:30.752236 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284850cc-e46a-4d6f-8099-82c038c2a163 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:30.761876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284850cc-e46a-4d6f-8099-82c038c2a163 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:30.782299 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b7543-9972-4c78-a5b2-46a2cdaed9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:30.783349 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b7543-9972-4c78-a5b2-46a2cdaed9fa admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:30.784021 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b7543-9972-4c78-a5b2-46a2cdaed9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:30.784391 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b7543-9972-4c78-a5b2-46a2cdaed9fa admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:30.787285 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830b7543-9972-4c78-a5b2-46a2cdaed9fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:30.802155 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830b7543-9972-4c78-a5b2-46a2cdaed9fa admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:30.811270 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 905/1812] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:00:30 2022] POST /identity/v3/users => generated 286 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:30.812559 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284850cc-e46a-4d6f-8099-82c038c2a163 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:30.836248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-284850cc-e46a-4d6f-8099-82c038c2a163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:30.852915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 908/1813] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:00:30 2022] POST /identity/v3/users => generated 297 bytes in 710 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:30.858348 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b7543-9972-4c78-a5b2-46a2cdaed9fa admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:30.880054 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284850cc-e46a-4d6f-8099-82c038c2a163 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:30.896234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b7543-9972-4c78-a5b2-46a2cdaed9fa admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:30.896426 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0639a-712c-4497-a8de-706869710986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:30.897113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0639a-712c-4497-a8de-706869710986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:30.897406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0639a-712c-4497-a8de-706869710986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:30.897922 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0639a-712c-4497-a8de-706869710986 admin admin] PATH_INFO: `/v3/projects/6c17ed3b3b9945da987db43b817faba1` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:30.900322 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0639a-712c-4497-a8de-706869710986 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c17ed3b3b9945da987db43b817faba1)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:30.902391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0639a-712c-4497-a8de-706869710986 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:30.909860 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf03515-c2e4-4ee8-939f-fb3bdab0c23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:30.913152 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf03515-c2e4-4ee8-939f-fb3bdab0c23f admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:30.914470 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf03515-c2e4-4ee8-939f-fb3bdab0c23f admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:30.915904 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf03515-c2e4-4ee8-939f-fb3bdab0c23f admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:30.919804 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf03515-c2e4-4ee8-939f-fb3bdab0c23f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:30.928006 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf03515-c2e4-4ee8-939f-fb3bdab0c23f admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:31.007465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 909/1814] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:31.016655 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf03515-c2e4-4ee8-939f-fb3bdab0c23f admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:31.089572 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-75f0639a-712c-4497-a8de-706869710986 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c17ed3b3b9945da987db43b817faba1', 'request_id': 'req-75f0639a-712c-4497-a8de-706869710986'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:31.090507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf03515-c2e4-4ee8-939f-fb3bdab0c23f admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:31.107900 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-75f0639a-712c-4497-a8de-706869710986 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c17ed3b3b9945da987db43b817faba1', 'request_id': 'req-75f0639a-712c-4497-a8de-706869710986'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:31.150714 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-75f0639a-712c-4497-a8de-706869710986 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c17ed3b3b9945da987db43b817faba1', 'request_id': 'req-75f0639a-712c-4497-a8de-706869710986'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:31.171015 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-75f0639a-712c-4497-a8de-706869710986 admin admin] The token cache is being invalidate because project 6c17ed3b3b9945da987db43b817faba1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:00:31.171378 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-75f0639a-712c-4497-a8de-706869710986 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:31.173763 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-75f0639a-712c-4497-a8de-706869710986 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:31.178795 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 906/1815] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:00:30 2022] DELETE /identity/v3/projects/6c17ed3b3b9945da987db43b817faba1 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:31.179807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0639a-712c-4497-a8de-706869710986 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:31.193244 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-75f0639a-712c-4497-a8de-706869710986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:31.201221 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:31.442613 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ca4c3-b40d-4899-809f-ce22ea120f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:31.443824 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ca4c3-b40d-4899-809f-ce22ea120f35 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:31.444318 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ca4c3-b40d-4899-809f-ce22ea120f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:31.444829 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ca4c3-b40d-4899-809f-ce22ea120f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:31.446082 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972ca4c3-b40d-4899-809f-ce22ea120f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:31.449536 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972ca4c3-b40d-4899-809f-ce22ea120f35 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:31.505465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 910/1816] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 498 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:31.524756 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a214fb-ff07-4989-b6cc-418619ab77e7 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:31.525353 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a214fb-ff07-4989-b6cc-418619ab77e7 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:31.525998 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a214fb-ff07-4989-b6cc-418619ab77e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:31.563953 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-d1a214fb-ff07-4989-b6cc-418619ab77e7 None None] MFA Rules not processed for user `dbdb8997891241e9b2e829feb8023f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:31.567086 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-d1a214fb-ff07-4989-b6cc-418619ab77e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:31.681309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0639a-712c-4497-a8de-706869710986 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:31.756710 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac501e58-9017-426d-8ce8-38e640e6b202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:31.766294 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac501e58-9017-426d-8ce8-38e640e6b202 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:31.774832 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac501e58-9017-426d-8ce8-38e640e6b202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:31.776385 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac501e58-9017-426d-8ce8-38e640e6b202 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:31.779847 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac501e58-9017-426d-8ce8-38e640e6b202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:31.789162 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac501e58-9017-426d-8ce8-38e640e6b202 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:31.825181 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 907/1817] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 646 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:31.833582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa2d3c-6984-41cc-81f1-8404723e452a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:31.834437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa2d3c-6984-41cc-81f1-8404723e452a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:31.835863 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa2d3c-6984-41cc-81f1-8404723e452a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:31.870494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 911/1818] 10.222.0.52 () {58 vars in 1015 bytes} [Tue Jul 26 05:00:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:31.879896 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff0c9b-96d3-468f-bd1e-0a9066d3acff None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:31.879896 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff0c9b-96d3-468f-bd1e-0a9066d3acff None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:31.880276 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff0c9b-96d3-468f-bd1e-0a9066d3acff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:31.943730 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3caa2d3c-6984-41cc-81f1-8404723e452a None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:31.943730 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3caa2d3c-6984-41cc-81f1-8404723e452a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:31.975487 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-4bff0c9b-96d3-468f-bd1e-0a9066d3acff None None] MFA Rules not processed for user `f0cc1af7d7d642889a1a11103f087e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:31.976949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-4bff0c9b-96d3-468f-bd1e-0a9066d3acff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:31.991322 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 908/1819] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:31.991322 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa2d3c-6984-41cc-81f1-8404723e452a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:31.991322 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3caa2d3c-6984-41cc-81f1-8404723e452a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:32.021975 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:32.048331 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa2d3c-6984-41cc-81f1-8404723e452a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:32.085036 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea0e99b-bc21-495f-82a3-89a1141faca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:32.085598 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0e99b-bc21-495f-82a3-89a1141faca0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:32.099832 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0e99b-bc21-495f-82a3-89a1141faca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:32.100852 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0e99b-bc21-495f-82a3-89a1141faca0 admin admin] PATH_INFO: `/v3/projects/145b5b1ebb8640fe80a2e52e766b2a69/users/6ae28a715af7496b981260452de90546/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:32.187995 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea0e99b-bc21-495f-82a3-89a1141faca0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=145b5b1ebb8640fe80a2e52e766b2a69, user_id=6ae28a715af7496b981260452de90546, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:32.187995 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea0e99b-bc21-495f-82a3-89a1141faca0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:32.209952 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 912/1820] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:32.215494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff0c9b-96d3-468f-bd1e-0a9066d3acff None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:32.227939 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 909/1821] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:31 2022] PUT /identity/v3/projects/145b5b1ebb8640fe80a2e52e766b2a69/users/6ae28a715af7496b981260452de90546/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:32.236268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d968e-fce6-4806-91ce-283f2b2cc995 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:32.236567 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d968e-fce6-4806-91ce-283f2b2cc995 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:32.236968 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d968e-fce6-4806-91ce-283f2b2cc995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:32.275493 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff0c9b-96d3-468f-bd1e-0a9066d3acff None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:32.330919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-011d968e-fce6-4806-91ce-283f2b2cc995 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:32.337583 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-011d968e-fce6-4806-91ce-283f2b2cc995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:32.556181 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 910/1822] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:32.556181 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d968e-fce6-4806-91ce-283f2b2cc995 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:32.560542 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-011d968e-fce6-4806-91ce-283f2b2cc995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:32.628207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ebb33-bfdf-4740-82ed-4031057fbc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:32.629182 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ebb33-bfdf-4740-82ed-4031057fbc01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:32.629550 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ebb33-bfdf-4740-82ed-4031057fbc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:32.629854 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ebb33-bfdf-4740-82ed-4031057fbc01 admin admin] PATH_INFO: `/v3/projects/34971f327bf546a8a0001c844a188141/users/812016e7e62d49c59bebe1ec24d376b2/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:32.748340 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ebb33-bfdf-4740-82ed-4031057fbc01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34971f327bf546a8a0001c844a188141, user_id=812016e7e62d49c59bebe1ec24d376b2, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:32.751484 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ebb33-bfdf-4740-82ed-4031057fbc01 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:32.796765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:32.828836 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 913/1823] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:32 2022] PUT /identity/v3/projects/34971f327bf546a8a0001c844a188141/users/812016e7e62d49c59bebe1ec24d376b2/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:32.843034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ebb33-bfdf-4740-82ed-4031057fbc01 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:32.878177 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d968e-fce6-4806-91ce-283f2b2cc995 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:33.099086 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a1b0e-9a45-4871-a846-dce61f028123 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:33.100437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a1b0e-9a45-4871-a846-dce61f028123 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:33.106084 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a1b0e-9a45-4871-a846-dce61f028123 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:33.107637 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a1b0e-9a45-4871-a846-dce61f028123 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:33.120177 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ebb33-bfdf-4740-82ed-4031057fbc01 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:33.142188 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf066e-8c00-4a55-a633-d206401b6f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:33.146299 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf066e-8c00-4a55-a633-d206401b6f68 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:33.147155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a1b0e-9a45-4871-a846-dce61f028123 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:33.148714 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf066e-8c00-4a55-a633-d206401b6f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:33.149314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf066e-8c00-4a55-a633-d206401b6f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:33.151969 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf066e-8c00-4a55-a633-d206401b6f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:33.157282 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a1b0e-9a45-4871-a846-dce61f028123 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:33.166013 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf066e-8c00-4a55-a633-d206401b6f68 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:33.253415 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 911/1824] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 699 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:33.265290 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 914/1825] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:33.265566 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a1b0e-9a45-4871-a846-dce61f028123 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:33.271796 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8f6a4-a3ab-4451-bc54-c63c898decad None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:33.273103 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8f6a4-a3ab-4451-bc54-c63c898decad None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:33.274580 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8f6a4-a3ab-4451-bc54-c63c898decad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:33.304039 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a1b0e-9a45-4871-a846-dce61f028123 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:33.320618 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69534b57-7489-4f70-8828-0918a53b4301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:33.321898 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69534b57-7489-4f70-8828-0918a53b4301 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:33.322418 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69534b57-7489-4f70-8828-0918a53b4301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:33.322918 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69534b57-7489-4f70-8828-0918a53b4301 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:33.324183 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69534b57-7489-4f70-8828-0918a53b4301 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:00:33.324807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69534b57-7489-4f70-8828-0918a53b4301 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:33.330198 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69534b57-7489-4f70-8828-0918a53b4301 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:33.367930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-6bc8f6a4-a3ab-4451-bc54-c63c898decad None None] MFA Rules not processed for user `812016e7e62d49c59bebe1ec24d376b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:33.371137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-6bc8f6a4-a3ab-4451-bc54-c63c898decad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:33.374429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 912/1826] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:00:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:00:33.375809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69534b57-7489-4f70-8828-0918a53b4301 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:33.418362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69534b57-7489-4f70-8828-0918a53b4301 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:33.451191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f920322-9162-43ca-b49e-5f7fe70e22c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:33.452630 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f920322-9162-43ca-b49e-5f7fe70e22c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:33.453807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f920322-9162-43ca-b49e-5f7fe70e22c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:33.454773 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f920322-9162-43ca-b49e-5f7fe70e22c0 admin admin] PATH_INFO: `/v3/projects/145b5b1ebb8640fe80a2e52e766b2a69/users/6ae28a715af7496b981260452de90546/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:33.463813 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f920322-9162-43ca-b49e-5f7fe70e22c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=145b5b1ebb8640fe80a2e52e766b2a69, user_id=6ae28a715af7496b981260452de90546, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:33.467789 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f920322-9162-43ca-b49e-5f7fe70e22c0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:33.521189 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 913/1827] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:33 2022] PUT /identity/v3/projects/145b5b1ebb8640fe80a2e52e766b2a69/users/6ae28a715af7496b981260452de90546/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:33.522535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f920322-9162-43ca-b49e-5f7fe70e22c0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:33.582153 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:33.593750 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 915/1828] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:33.598633 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baaff4c-543f-4125-9acb-4944e6dd3ec7 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:33.599889 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baaff4c-543f-4125-9acb-4944e6dd3ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:33.601076 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baaff4c-543f-4125-9acb-4944e6dd3ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:33.713478 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-1baaff4c-543f-4125-9acb-4944e6dd3ec7 None None] MFA Rules not processed for user `6ae28a715af7496b981260452de90546`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:33.717594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-1baaff4c-543f-4125-9acb-4944e6dd3ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:33.792248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f920322-9162-43ca-b49e-5f7fe70e22c0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:33.815771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f38f4-1595-4b3b-9173-a425f474df5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:33.816988 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f38f4-1595-4b3b-9173-a425f474df5b admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:33.817516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f38f4-1595-4b3b-9173-a425f474df5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:33.817975 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f38f4-1595-4b3b-9173-a425f474df5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:33.818988 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f38f4-1595-4b3b-9173-a425f474df5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:33.825499 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f38f4-1595-4b3b-9173-a425f474df5b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:33.931049 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 914/1829] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:00:33 2022] POST /identity/v3/projects => generated 367 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:33.936919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f38f4-1595-4b3b-9173-a425f474df5b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:33.947889 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 916/1830] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:33.948987 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baaff4c-543f-4125-9acb-4944e6dd3ec7 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:33.980722 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baaff4c-543f-4125-9acb-4944e6dd3ec7 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:34.012399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:34.143973 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a75622-dd69-46fe-911e-19f8e555c560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:34.144901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a75622-dd69-46fe-911e-19f8e555c560 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:34.145144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a75622-dd69-46fe-911e-19f8e555c560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:34.145430 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a75622-dd69-46fe-911e-19f8e555c560 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:34.146507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a75622-dd69-46fe-911e-19f8e555c560 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:34.150289 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a75622-dd69-46fe-911e-19f8e555c560 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:34.206549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f38f4-1595-4b3b-9173-a425f474df5b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:34.224422 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96495e-288a-48ae-9bee-e6880438daa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:34.225095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96495e-288a-48ae-9bee-e6880438daa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:34.225408 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96495e-288a-48ae-9bee-e6880438daa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:34.225649 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96495e-288a-48ae-9bee-e6880438daa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:34.226371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af96495e-288a-48ae-9bee-e6880438daa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:34.228417 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af96495e-288a-48ae-9bee-e6880438daa2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:34.243757 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 917/1831] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:00:33 2022] POST /identity/v3/users => generated 284 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:34.243757 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a75622-dd69-46fe-911e-19f8e555c560 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:34.250328 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-f8a75622-dd69-46fe-911e-19f8e555c560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:34.318521 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 915/1832] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:00:33 2022] POST /identity/v3/projects => generated 371 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:34.319937 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96495e-288a-48ae-9bee-e6880438daa2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:34.592451 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a75622-dd69-46fe-911e-19f8e555c560 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:34.602463 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96495e-288a-48ae-9bee-e6880438daa2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:34.644395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215d5e1-78c1-4c38-b429-2d2fa097ee75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:34.645483 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215d5e1-78c1-4c38-b429-2d2fa097ee75 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:34.650166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215d5e1-78c1-4c38-b429-2d2fa097ee75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:34.650964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215d5e1-78c1-4c38-b429-2d2fa097ee75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:34.652049 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7215d5e1-78c1-4c38-b429-2d2fa097ee75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:34.655296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7215d5e1-78c1-4c38-b429-2d2fa097ee75 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:34.700273 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 916/1833] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:34.704463 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215d5e1-78c1-4c38-b429-2d2fa097ee75 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:34.760604 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215d5e1-78c1-4c38-b429-2d2fa097ee75 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:34.790273 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec390cec-fd74-4c5d-bad4-f6cf98d70eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:34.790927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec390cec-fd74-4c5d-bad4-f6cf98d70eaa admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:34.791227 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec390cec-fd74-4c5d-bad4-f6cf98d70eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:34.791570 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec390cec-fd74-4c5d-bad4-f6cf98d70eaa admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:34.792336 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec390cec-fd74-4c5d-bad4-f6cf98d70eaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:34.796302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec390cec-fd74-4c5d-bad4-f6cf98d70eaa admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:34.838122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 917/1834] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:00:34 2022] POST /identity/v3/users => generated 292 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:34.849998 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec390cec-fd74-4c5d-bad4-f6cf98d70eaa admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:34.867942 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:34.886331 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec390cec-fd74-4c5d-bad4-f6cf98d70eaa admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:34.905633 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aaf771-3088-4d71-aad0-f183aabc442b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:34.906331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aaf771-3088-4d71-aad0-f183aabc442b service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:34.906580 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aaf771-3088-4d71-aad0-f183aabc442b service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:34.906862 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aaf771-3088-4d71-aad0-f183aabc442b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:34.917591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c5afd-5b71-4926-abda-88c003724daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:34.918272 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c5afd-5b71-4926-abda-88c003724daf admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:34.920887 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c5afd-5b71-4926-abda-88c003724daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:34.921283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c5afd-5b71-4926-abda-88c003724daf admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:34.922083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c5afd-5b71-4926-abda-88c003724daf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:34.926659 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c5afd-5b71-4926-abda-88c003724daf admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:34.968693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 918/1835] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:34.970408 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c5afd-5b71-4926-abda-88c003724daf admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:35.017695 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c5afd-5b71-4926-abda-88c003724daf admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:35.034682 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf5ff4-1c34-4e63-a950-d7ac911b5657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:35.034682 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf5ff4-1c34-4e63-a950-d7ac911b5657 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:35.034682 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf5ff4-1c34-4e63-a950-d7ac911b5657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:35.034682 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf5ff4-1c34-4e63-a950-d7ac911b5657 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:35.037038 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf5ff4-1c34-4e63-a950-d7ac911b5657 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:35.037399 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf5ff4-1c34-4e63-a950-d7ac911b5657 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:35.059423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 919/1836] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:35.060948 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf5ff4-1c34-4e63-a950-d7ac911b5657 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:35.117582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf5ff4-1c34-4e63-a950-d7ac911b5657 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:35.147456 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be31be1-f1a5-4599-803c-091ab24a065d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:35.148227 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be31be1-f1a5-4599-803c-091ab24a065d admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:35.149202 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be31be1-f1a5-4599-803c-091ab24a065d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:35.149494 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be31be1-f1a5-4599-803c-091ab24a065d admin admin] PATH_INFO: `/v3/projects/a3acdb67fcff463695c57f69b1e5a54d/users/04df43d589a545c1ade3a31129134bb4/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:35.274655 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be31be1-f1a5-4599-803c-091ab24a065d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3acdb67fcff463695c57f69b1e5a54d, user_id=04df43d589a545c1ade3a31129134bb4, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:35.279639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aaf771-3088-4d71-aad0-f183aabc442b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:35.282533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aaf771-3088-4d71-aad0-f183aabc442b service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:35.297095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be31be1-f1a5-4599-803c-091ab24a065d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:35.360174 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 918/1837] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 1120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:35.367852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aaf771-3088-4d71-aad0-f183aabc442b service cinder] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:35.383616 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 920/1838] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:35 2022] PUT /identity/v3/projects/a3acdb67fcff463695c57f69b1e5a54d/users/04df43d589a545c1ade3a31129134bb4/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:35.408761 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1481191-cd65-4044-a7e6-6954e1fb5037 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:35.409125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1481191-cd65-4044-a7e6-6954e1fb5037 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:35.409539 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1481191-cd65-4044-a7e6-6954e1fb5037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:35.454267 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aaf771-3088-4d71-aad0-f183aabc442b service cinder] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:35.478629 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:35.573709 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e1481191-cd65-4044-a7e6-6954e1fb5037 None None] MFA Rules not processed for user `04df43d589a545c1ade3a31129134bb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:35.577995 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e1481191-cd65-4044-a7e6-6954e1fb5037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:35.831971 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 921/1839] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:35 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 425 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:35.833125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1481191-cd65-4044-a7e6-6954e1fb5037 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:35.860668 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac727e9a-274c-4174-8f49-fd38159cc061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:35.861375 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac727e9a-274c-4174-8f49-fd38159cc061 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:35.861657 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac727e9a-274c-4174-8f49-fd38159cc061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:35.861934 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac727e9a-274c-4174-8f49-fd38159cc061 admin admin] PATH_INFO: `/v3/projects/b60c24c2fcaa4e599e8e925eeaf31006/users/6aaa8c50971d43cbb4b6f970f5ad18d2/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:35.870546 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1481191-cd65-4044-a7e6-6954e1fb5037 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:35.890758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476de04-49fb-4b1f-bf9d-f160bd255a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:35.891505 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476de04-49fb-4b1f-bf9d-f160bd255a6a admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:35.891819 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476de04-49fb-4b1f-bf9d-f160bd255a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:35.892297 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476de04-49fb-4b1f-bf9d-f160bd255a6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:35.892846 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2476de04-49fb-4b1f-bf9d-f160bd255a6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:35.894932 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2476de04-49fb-4b1f-bf9d-f160bd255a6a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:35.920452 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac727e9a-274c-4174-8f49-fd38159cc061 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b60c24c2fcaa4e599e8e925eeaf31006, user_id=6aaa8c50971d43cbb4b6f970f5ad18d2, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:35.924596 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac727e9a-274c-4174-8f49-fd38159cc061 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:36.000263 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 922/1840] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:00:35 2022] POST /identity/v3/projects => generated 369 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:36.001379 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 919/1841] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:35 2022] PUT /identity/v3/projects/b60c24c2fcaa4e599e8e925eeaf31006/users/6aaa8c50971d43cbb4b6f970f5ad18d2/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:36.001489 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476de04-49fb-4b1f-bf9d-f160bd255a6a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:36.002469 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac727e9a-274c-4174-8f49-fd38159cc061 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:36.013152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2476de04-49fb-4b1f-bf9d-f160bd255a6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:36.367045 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476de04-49fb-4b1f-bf9d-f160bd255a6a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:36.376562 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac727e9a-274c-4174-8f49-fd38159cc061 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:36.388867 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e43ca-a905-4cd1-aba9-a74b2f55e45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:36.389575 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e43ca-a905-4cd1-aba9-a74b2f55e45f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:36.389858 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e43ca-a905-4cd1-aba9-a74b2f55e45f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:36.390149 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e43ca-a905-4cd1-aba9-a74b2f55e45f admin admin] PATH_INFO: `/v3/users/c6112f9979c94d5aa6f8756bd1335479` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:36.391452 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26cf3a-e581-4ffe-8d36-079568d777ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:36.392287 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26cf3a-e581-4ffe-8d36-079568d777ce admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:36.392582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26cf3a-e581-4ffe-8d36-079568d777ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:36.392862 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26cf3a-e581-4ffe-8d36-079568d777ce admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:36.393575 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26cf3a-e581-4ffe-8d36-079568d777ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:36.404903 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e43ca-a905-4cd1-aba9-a74b2f55e45f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6112f9979c94d5aa6f8756bd1335479)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:36.408165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26cf3a-e581-4ffe-8d36-079568d777ce admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:36.412181 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e43ca-a905-4cd1-aba9-a74b2f55e45f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:36.486204 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 920/1842] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:00:36 2022] POST /identity/v3/users => generated 291 bytes in 485 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:36.489024 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26cf3a-e581-4ffe-8d36-079568d777ce admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:36.518104 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26cf3a-e581-4ffe-8d36-079568d777ce admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:36.521724 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:36.530572 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270b7e2-4c96-49c8-8dca-55ecfdc08548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:36.532995 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270b7e2-4c96-49c8-8dca-55ecfdc08548 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:36.533462 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270b7e2-4c96-49c8-8dca-55ecfdc08548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:36.535741 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270b7e2-4c96-49c8-8dca-55ecfdc08548 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:36.549063 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270b7e2-4c96-49c8-8dca-55ecfdc08548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:36.550892 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270b7e2-4c96-49c8-8dca-55ecfdc08548 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:36.589978 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 921/1843] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:36.590964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f20e43ca-a905-4cd1-aba9-a74b2f55e45f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6112f9979c94d5aa6f8756bd1335479'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:36.591157 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270b7e2-4c96-49c8-8dca-55ecfdc08548 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:36.616449 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270b7e2-4c96-49c8-8dca-55ecfdc08548 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:36.642368 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc78fc7-ad5e-4209-8067-3a151c7023c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:36.643206 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc78fc7-ad5e-4209-8067-3a151c7023c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:36.643513 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc78fc7-ad5e-4209-8067-3a151c7023c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:36.643845 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc78fc7-ad5e-4209-8067-3a151c7023c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:36.645031 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc78fc7-ad5e-4209-8067-3a151c7023c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:36.647290 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f20e43ca-a905-4cd1-aba9-a74b2f55e45f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6112f9979c94d5aa6f8756bd1335479'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:36.651522 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f20e43ca-a905-4cd1-aba9-a74b2f55e45f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6112f9979c94d5aa6f8756bd1335479'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:36.653517 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc78fc7-ad5e-4209-8067-3a151c7023c4 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:36.693432 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 922/1844] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:36.693432 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc78fc7-ad5e-4209-8067-3a151c7023c4 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:36.694197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f20e43ca-a905-4cd1-aba9-a74b2f55e45f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6112f9979c94d5aa6f8756bd1335479'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:36.698605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-8cc78fc7-ad5e-4209-8067-3a151c7023c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:36.699482 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f20e43ca-a905-4cd1-aba9-a74b2f55e45f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6112f9979c94d5aa6f8756bd1335479'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:36.739149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc78fc7-ad5e-4209-8067-3a151c7023c4 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:36.758951 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6097ef-c651-4004-b11f-c728eb7ee299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:36.759702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6097ef-c651-4004-b11f-c728eb7ee299 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:36.759992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6097ef-c651-4004-b11f-c728eb7ee299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:36.760302 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6097ef-c651-4004-b11f-c728eb7ee299 admin admin] PATH_INFO: `/v3/projects/b60c24c2fcaa4e599e8e925eeaf31006/users/6aaa8c50971d43cbb4b6f970f5ad18d2/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:36.765772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6097ef-c651-4004-b11f-c728eb7ee299 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b60c24c2fcaa4e599e8e925eeaf31006, user_id=6aaa8c50971d43cbb4b6f970f5ad18d2, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:36.768226 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6097ef-c651-4004-b11f-c728eb7ee299 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:36.804584 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 923/1845] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:36 2022] PUT /identity/v3/projects/b60c24c2fcaa4e599e8e925eeaf31006/users/6aaa8c50971d43cbb4b6f970f5ad18d2/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:36.804584 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6097ef-c651-4004-b11f-c728eb7ee299 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:36.809553 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-3a6097ef-c651-4004-b11f-c728eb7ee299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:36.872754 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 923/1846] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:00:35 2022] DELETE /identity/v3/users/c6112f9979c94d5aa6f8756bd1335479 => generated 0 bytes in 875 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:36.876868 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288194f-753d-4640-8f92-4cfe3bd52aa6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:36.877672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288194f-753d-4640-8f92-4cfe3bd52aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:36.881223 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288194f-753d-4640-8f92-4cfe3bd52aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:36.932581 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:37.002871 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6097ef-c651-4004-b11f-c728eb7ee299 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:37.032428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7288194f-753d-4640-8f92-4cfe3bd52aa6 None None] MFA Rules not processed for user `6aaa8c50971d43cbb4b6f970f5ad18d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:37.039514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7288194f-753d-4640-8f92-4cfe3bd52aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:37.322342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 924/1847] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:36 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 449 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:37.330282 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d11476-90f0-4d60-8d07-1c4d65bee8e7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:37.330983 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d11476-90f0-4d60-8d07-1c4d65bee8e7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:37.332413 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d11476-90f0-4d60-8d07-1c4d65bee8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:37.401625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d987bbd-e9f3-4d9f-af94-fa6af1f36b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:37.403194 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d987bbd-e9f3-4d9f-af94-fa6af1f36b08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:37.403781 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d987bbd-e9f3-4d9f-af94-fa6af1f36b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:37.413503 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d987bbd-e9f3-4d9f-af94-fa6af1f36b08 admin admin] PATH_INFO: `/v3/projects/f8098c6aff364cb89a5f77a55a3a1029/users/e5dbc643b386409985631ef6ecbb7bdf/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:37.470843 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-34d11476-90f0-4d60-8d07-1c4d65bee8e7 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:37.473515 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-34d11476-90f0-4d60-8d07-1c4d65bee8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:37.535041 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 925/1848] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:37.542981 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d987bbd-e9f3-4d9f-af94-fa6af1f36b08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8098c6aff364cb89a5f77a55a3a1029, user_id=e5dbc643b386409985631ef6ecbb7bdf, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:37.547094 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d11476-90f0-4d60-8d07-1c4d65bee8e7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:37.557712 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d987bbd-e9f3-4d9f-af94-fa6af1f36b08 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:37.562273 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-34d11476-90f0-4d60-8d07-1c4d65bee8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:37.641031 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 924/1849] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:36 2022] PUT /identity/v3/projects/f8098c6aff364cb89a5f77a55a3a1029/users/e5dbc643b386409985631ef6ecbb7bdf/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 844 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:37.644920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d11476-90f0-4d60-8d07-1c4d65bee8e7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:37.650606 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d987bbd-e9f3-4d9f-af94-fa6af1f36b08 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:37.665317 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-3d987bbd-e9f3-4d9f-af94-fa6af1f36b08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:37.920988 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b5e231-9f52-40fd-a676-69f94d2d3da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:37.921705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5e231-9f52-40fd-a676-69f94d2d3da6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:37.922007 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5e231-9f52-40fd-a676-69f94d2d3da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:37.922292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5e231-9f52-40fd-a676-69f94d2d3da6 admin admin] PATH_INFO: `/v3/users/69e8fcabe6f74fc2992b7bd94a4310f8` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:37.928588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b5e231-9f52-40fd-a676-69f94d2d3da6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69e8fcabe6f74fc2992b7bd94a4310f8)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:37.930727 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b5e231-9f52-40fd-a676-69f94d2d3da6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:38.000598 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:38.034502 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d987bbd-e9f3-4d9f-af94-fa6af1f36b08 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:38.055282 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-12b5e231-9f52-40fd-a676-69f94d2d3da6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69e8fcabe6f74fc2992b7bd94a4310f8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:38.059326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e266dd2-690a-45aa-86ce-619465d50127 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:38.063275 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e266dd2-690a-45aa-86ce-619465d50127 service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:38.065113 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e266dd2-690a-45aa-86ce-619465d50127 service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:38.068043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e266dd2-690a-45aa-86ce-619465d50127 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:38.074169 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-12b5e231-9f52-40fd-a676-69f94d2d3da6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69e8fcabe6f74fc2992b7bd94a4310f8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:38.096166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-12b5e231-9f52-40fd-a676-69f94d2d3da6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69e8fcabe6f74fc2992b7bd94a4310f8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:38.108327 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-12b5e231-9f52-40fd-a676-69f94d2d3da6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69e8fcabe6f74fc2992b7bd94a4310f8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:38.113029 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-12b5e231-9f52-40fd-a676-69f94d2d3da6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69e8fcabe6f74fc2992b7bd94a4310f8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:38.185590 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 926/1850] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:00:37 2022] DELETE /identity/v3/users/69e8fcabe6f74fc2992b7bd94a4310f8 => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:38.204767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b5e231-9f52-40fd-a676-69f94d2d3da6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:38.213350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-12b5e231-9f52-40fd-a676-69f94d2d3da6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:38.381583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e266dd2-690a-45aa-86ce-619465d50127 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:38.389148 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e266dd2-690a-45aa-86ce-619465d50127 service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:38.433728 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 925/1851] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:37 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 792 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:38.440318 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e266dd2-690a-45aa-86ce-619465d50127 service cinder] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:38.447502 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-7e266dd2-690a-45aa-86ce-619465d50127 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:38.611239 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:38.664229 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b5e231-9f52-40fd-a676-69f94d2d3da6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:38.688631 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5114977-7c79-4d86-9d9c-a38e587f1deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:38.689269 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5114977-7c79-4d86-9d9c-a38e587f1deb admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:38.689581 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5114977-7c79-4d86-9d9c-a38e587f1deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:38.689876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5114977-7c79-4d86-9d9c-a38e587f1deb admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:38.690694 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5114977-7c79-4d86-9d9c-a38e587f1deb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:38.696738 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5114977-7c79-4d86-9d9c-a38e587f1deb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:38.750209 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e266dd2-690a-45aa-86ce-619465d50127 service cinder] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:38.779372 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 927/1852] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 589 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:38.794944 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b0777-be8c-4047-bbc1-a53b0c89c633 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:38.795763 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b0777-be8c-4047-bbc1-a53b0c89c633 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:38.796341 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b0777-be8c-4047-bbc1-a53b0c89c633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:38.922962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16309fc9-dae1-4224-b186-6c4e93c2049f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:38.925013 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16309fc9-dae1-4224-b186-6c4e93c2049f service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:38.925297 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16309fc9-dae1-4224-b186-6c4e93c2049f service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:38.925807 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16309fc9-dae1-4224-b186-6c4e93c2049f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:38.926141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-504b0777-be8c-4047-bbc1-a53b0c89c633 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:38.933424 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-504b0777-be8c-4047-bbc1-a53b0c89c633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:38.934958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-16309fc9-dae1-4224-b186-6c4e93c2049f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:38.969073 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 928/1853] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:38.971267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b0777-be8c-4047-bbc1-a53b0c89c633 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:39.007726 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16309fc9-dae1-4224-b186-6c4e93c2049f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:39.011121 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16309fc9-dae1-4224-b186-6c4e93c2049f service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:39.020187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b0777-be8c-4047-bbc1-a53b0c89c633 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:39.060574 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095c4ef-0ed7-40ef-9dce-bfbdb697d83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:39.061884 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095c4ef-0ed7-40ef-9dce-bfbdb697d83c admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:39.062396 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095c4ef-0ed7-40ef-9dce-bfbdb697d83c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:39.062929 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095c4ef-0ed7-40ef-9dce-bfbdb697d83c admin admin] PATH_INFO: `/v3/projects/f8098c6aff364cb89a5f77a55a3a1029/users/e5dbc643b386409985631ef6ecbb7bdf/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:39.075276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5095c4ef-0ed7-40ef-9dce-bfbdb697d83c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8098c6aff364cb89a5f77a55a3a1029, user_id=e5dbc643b386409985631ef6ecbb7bdf, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:39.089265 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5095c4ef-0ed7-40ef-9dce-bfbdb697d83c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:39.100197 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 926/1854] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:39.104122 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16309fc9-dae1-4224-b186-6c4e93c2049f service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:39.115857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-16309fc9-dae1-4224-b186-6c4e93c2049f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:39.172284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 929/1855] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:38 2022] PUT /identity/v3/projects/f8098c6aff364cb89a5f77a55a3a1029/users/e5dbc643b386409985631ef6ecbb7bdf/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:39.182969 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff176695-1ac8-48b4-b4a9-f31ef44bfa18 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:39.183451 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff176695-1ac8-48b4-b4a9-f31ef44bfa18 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:39.184051 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff176695-1ac8-48b4-b4a9-f31ef44bfa18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:39.285465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ff176695-1ac8-48b4-b4a9-f31ef44bfa18 None None] MFA Rules not processed for user `e5dbc643b386409985631ef6ecbb7bdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:39.287363 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ff176695-1ac8-48b4-b4a9-f31ef44bfa18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:39.293481 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16309fc9-dae1-4224-b186-6c4e93c2049f service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:39.578325 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 930/1856] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:39 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:39.578907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d522d-951e-4739-bf5f-6fa80f46c84f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:39.580793 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff176695-1ac8-48b4-b4a9-f31ef44bfa18 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:39.582163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d522d-951e-4739-bf5f-6fa80f46c84f service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:39.583177 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d522d-951e-4739-bf5f-6fa80f46c84f service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:39.585124 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ff176695-1ac8-48b4-b4a9-f31ef44bfa18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:39.585436 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d522d-951e-4739-bf5f-6fa80f46c84f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:39.592845 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-107d522d-951e-4739-bf5f-6fa80f46c84f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:39.603759 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:39.765245 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff176695-1ac8-48b4-b4a9-f31ef44bfa18 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:39.772191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc105d-a548-48e8-9ab4-deedff5b13a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:39.772938 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc105d-a548-48e8-9ab4-deedff5b13a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:39.773227 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc105d-a548-48e8-9ab4-deedff5b13a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:39.773515 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc105d-a548-48e8-9ab4-deedff5b13a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:39.881286 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:39.900001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107d522d-951e-4739-bf5f-6fa80f46c84f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:39.905895 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107d522d-951e-4739-bf5f-6fa80f46c84f service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:39.949771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc105d-a548-48e8-9ab4-deedff5b13a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:39.953267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc105d-a548-48e8-9ab4-deedff5b13a8 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:39.962833 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 927/1857] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 861 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:39.964698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d522d-951e-4739-bf5f-6fa80f46c84f service cinder] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:39.971507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-107d522d-951e-4739-bf5f-6fa80f46c84f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:39.997086 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 931/1858] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:40.021256 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d522d-951e-4739-bf5f-6fa80f46c84f service cinder] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:40.047910 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5149634-d63e-4d20-81d2-a46c250e3afc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:40.052172 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5149634-d63e-4d20-81d2-a46c250e3afc service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:40.052506 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5149634-d63e-4d20-81d2-a46c250e3afc service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:40.052981 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5149634-d63e-4d20-81d2-a46c250e3afc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:40.109566 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5149634-d63e-4d20-81d2-a46c250e3afc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:40.115359 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5149634-d63e-4d20-81d2-a46c250e3afc service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:40.158572 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 928/1859] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:39 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:41.129642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820d342-6a89-47fc-887c-64b5315086af None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:41.130459 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820d342-6a89-47fc-887c-64b5315086af None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:41.130459 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820d342-6a89-47fc-887c-64b5315086af None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:41.132027 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 932/1860] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jul 26 05:00:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 05:00:41.166421 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a32c88-41f8-4ec3-9810-22e6e8d0f2dc None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:41.167313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a32c88-41f8-4ec3-9810-22e6e8d0f2dc None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:41.167313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a32c88-41f8-4ec3-9810-22e6e8d0f2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:41.221592 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-29a32c88-41f8-4ec3-9810-22e6e8d0f2dc None None] MFA Rules not processed for user `2faeed9002ee4154bc980c146cd06bcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:41.224890 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-29a32c88-41f8-4ec3-9810-22e6e8d0f2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:41.419172 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 929/1861] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jul 26 05:00:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:41.708261 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820d342-6a89-47fc-887c-64b5315086af None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:41.715235 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8820d342-6a89-47fc-887c-64b5315086af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:41.743731 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820d342-6a89-47fc-887c-64b5315086af None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:41.754305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522aa447-aadb-4fb9-826c-11580a37332b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:41.760633 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522aa447-aadb-4fb9-826c-11580a37332b service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:41.774354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522aa447-aadb-4fb9-826c-11580a37332b service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:41.774640 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522aa447-aadb-4fb9-826c-11580a37332b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:41.793733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522aa447-aadb-4fb9-826c-11580a37332b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:41.796853 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522aa447-aadb-4fb9-826c-11580a37332b service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:41.821436 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 933/1862] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:42.659858 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a32c88-41f8-4ec3-9810-22e6e8d0f2dc None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:42.662488 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-29a32c88-41f8-4ec3-9810-22e6e8d0f2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:42.687686 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a32c88-41f8-4ec3-9810-22e6e8d0f2dc None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:42.697237 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf886fa6-1a99-4266-a9a5-65269daedb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:42.698169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf886fa6-1a99-4266-a9a5-65269daedb82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:42.698608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf886fa6-1a99-4266-a9a5-65269daedb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:42.699032 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf886fa6-1a99-4266-a9a5-65269daedb82 admin admin] PATH_INFO: `/v3/projects/ce5a4b3e5a7f4c15b0b1ca7758779773` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:42.700634 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf886fa6-1a99-4266-a9a5-65269daedb82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce5a4b3e5a7f4c15b0b1ca7758779773)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:42.703938 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf886fa6-1a99-4266-a9a5-65269daedb82 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:42.853268 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-cf886fa6-1a99-4266-a9a5-65269daedb82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce5a4b3e5a7f4c15b0b1ca7758779773', 'request_id': 'req-cf886fa6-1a99-4266-a9a5-65269daedb82'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:42.871833 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-cf886fa6-1a99-4266-a9a5-65269daedb82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce5a4b3e5a7f4c15b0b1ca7758779773', 'request_id': 'req-cf886fa6-1a99-4266-a9a5-65269daedb82'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:42.887586 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-cf886fa6-1a99-4266-a9a5-65269daedb82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce5a4b3e5a7f4c15b0b1ca7758779773', 'request_id': 'req-cf886fa6-1a99-4266-a9a5-65269daedb82'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:42.923050 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-cf886fa6-1a99-4266-a9a5-65269daedb82 admin admin] The token cache is being invalidate because project ce5a4b3e5a7f4c15b0b1ca7758779773 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:00:42.926441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-cf886fa6-1a99-4266-a9a5-65269daedb82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:42.927920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-cf886fa6-1a99-4266-a9a5-65269daedb82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:42.932318 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 930/1863] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:00:42 2022] DELETE /identity/v3/projects/ce5a4b3e5a7f4c15b0b1ca7758779773 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:42.938632 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522aa447-aadb-4fb9-826c-11580a37332b service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:42.962886 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522aa447-aadb-4fb9-826c-11580a37332b service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:43.153185 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da9ba3-78c6-497c-acb3-865a32e043e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:43.154268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da9ba3-78c6-497c-acb3-865a32e043e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:43.154984 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da9ba3-78c6-497c-acb3-865a32e043e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:43.155313 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da9ba3-78c6-497c-acb3-865a32e043e8 admin admin] PATH_INFO: `/v3/users/15eddeafa63e4feca20246f7eb9712fc` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:43.157159 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da9ba3-78c6-497c-acb3-865a32e043e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15eddeafa63e4feca20246f7eb9712fc)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:43.159383 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da9ba3-78c6-497c-acb3-865a32e043e8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:43.258897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:43.269697 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-12da9ba3-78c6-497c-acb3-865a32e043e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15eddeafa63e4feca20246f7eb9712fc'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:43.292752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-12da9ba3-78c6-497c-acb3-865a32e043e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15eddeafa63e4feca20246f7eb9712fc'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:43.293945 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-12da9ba3-78c6-497c-acb3-865a32e043e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15eddeafa63e4feca20246f7eb9712fc'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:43.324180 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-12da9ba3-78c6-497c-acb3-865a32e043e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15eddeafa63e4feca20246f7eb9712fc'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:43.325916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-12da9ba3-78c6-497c-acb3-865a32e043e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15eddeafa63e4feca20246f7eb9712fc'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:43.395738 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 934/1864] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:00:42 2022] DELETE /identity/v3/users/15eddeafa63e4feca20246f7eb9712fc => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:43.640592 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf886fa6-1a99-4266-a9a5-65269daedb82 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:43.644932 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-cf886fa6-1a99-4266-a9a5-65269daedb82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:43.678438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf886fa6-1a99-4266-a9a5-65269daedb82 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:43.770446 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da9ba3-78c6-497c-acb3-865a32e043e8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:43.777355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-12da9ba3-78c6-497c-acb3-865a32e043e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:43.819404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:43.836339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2fd1c3-2f92-4ce5-a3e1-bf2276ec88e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:43.837837 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2fd1c3-2f92-4ce5-a3e1-bf2276ec88e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:43.838313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2fd1c3-2f92-4ce5-a3e1-bf2276ec88e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:43.838890 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2fd1c3-2f92-4ce5-a3e1-bf2276ec88e0 admin admin] PATH_INFO: `/v3/projects/d6a6c2ad05e94d8c9bed191360d80c87` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:43.844365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2fd1c3-2f92-4ce5-a3e1-bf2276ec88e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6a6c2ad05e94d8c9bed191360d80c87)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:43.849823 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2fd1c3-2f92-4ce5-a3e1-bf2276ec88e0 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:43.988331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8c2fd1c3-2f92-4ce5-a3e1-bf2276ec88e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6a6c2ad05e94d8c9bed191360d80c87', 'request_id': 'req-8c2fd1c3-2f92-4ce5-a3e1-bf2276ec88e0'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:44.019357 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8c2fd1c3-2f92-4ce5-a3e1-bf2276ec88e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6a6c2ad05e94d8c9bed191360d80c87', 'request_id': 'req-8c2fd1c3-2f92-4ce5-a3e1-bf2276ec88e0'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:44.025847 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da9ba3-78c6-497c-acb3-865a32e043e8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:44.062772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8c2fd1c3-2f92-4ce5-a3e1-bf2276ec88e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6a6c2ad05e94d8c9bed191360d80c87', 'request_id': 'req-8c2fd1c3-2f92-4ce5-a3e1-bf2276ec88e0'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:44.102756 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76847f87-64dc-4bdc-9ed0-5199032b30c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:44.103369 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76847f87-64dc-4bdc-9ed0-5199032b30c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:44.104046 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76847f87-64dc-4bdc-9ed0-5199032b30c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:44.104373 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76847f87-64dc-4bdc-9ed0-5199032b30c6 admin admin] PATH_INFO: `/v3/users/04df43d589a545c1ade3a31129134bb4` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:44.105710 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76847f87-64dc-4bdc-9ed0-5199032b30c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04df43d589a545c1ade3a31129134bb4)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:44.108720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76847f87-64dc-4bdc-9ed0-5199032b30c6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:44.134018 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8c2fd1c3-2f92-4ce5-a3e1-bf2276ec88e0 admin admin] The token cache is being invalidate because project d6a6c2ad05e94d8c9bed191360d80c87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:00:44.134489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8c2fd1c3-2f92-4ce5-a3e1-bf2276ec88e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:44.135327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8c2fd1c3-2f92-4ce5-a3e1-bf2276ec88e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:44.137252 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 931/1865] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:00:43 2022] DELETE /identity/v3/projects/d6a6c2ad05e94d8c9bed191360d80c87 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:44.141661 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2fd1c3-2f92-4ce5-a3e1-bf2276ec88e0 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:44.143432 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-8c2fd1c3-2f92-4ce5-a3e1-bf2276ec88e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:44.169767 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2fd1c3-2f92-4ce5-a3e1-bf2276ec88e0 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:44.182750 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54382182-bb9f-4090-98cb-32fad989b2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:44.183460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54382182-bb9f-4090-98cb-32fad989b2e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:44.183791 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54382182-bb9f-4090-98cb-32fad989b2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:44.184215 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54382182-bb9f-4090-98cb-32fad989b2e7 admin admin] PATH_INFO: `/v3/users/6ae28a715af7496b981260452de90546` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:44.186622 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54382182-bb9f-4090-98cb-32fad989b2e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ae28a715af7496b981260452de90546)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:44.194822 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54382182-bb9f-4090-98cb-32fad989b2e7 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:44.198662 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-76847f87-64dc-4bdc-9ed0-5199032b30c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04df43d589a545c1ade3a31129134bb4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:44.218728 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-76847f87-64dc-4bdc-9ed0-5199032b30c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04df43d589a545c1ade3a31129134bb4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:44.222859 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-76847f87-64dc-4bdc-9ed0-5199032b30c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04df43d589a545c1ade3a31129134bb4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:44.239726 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-76847f87-64dc-4bdc-9ed0-5199032b30c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04df43d589a545c1ade3a31129134bb4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:44.241884 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-76847f87-64dc-4bdc-9ed0-5199032b30c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04df43d589a545c1ade3a31129134bb4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:44.249025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:44.272066 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 935/1866] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:00:43 2022] DELETE /identity/v3/users/04df43d589a545c1ade3a31129134bb4 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:44.277426 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a4a30-344f-4b60-8299-698ac424aae1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:44.277948 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a4a30-344f-4b60-8299-698ac424aae1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:44.278347 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a4a30-344f-4b60-8299-698ac424aae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:44.303087 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-54382182-bb9f-4090-98cb-32fad989b2e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ae28a715af7496b981260452de90546'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:44.316350 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-54382182-bb9f-4090-98cb-32fad989b2e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ae28a715af7496b981260452de90546'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:44.319739 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-708a4a30-344f-4b60-8299-698ac424aae1 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:44.324182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-708a4a30-344f-4b60-8299-698ac424aae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:44.348485 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-54382182-bb9f-4090-98cb-32fad989b2e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ae28a715af7496b981260452de90546'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:44.350508 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-54382182-bb9f-4090-98cb-32fad989b2e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ae28a715af7496b981260452de90546'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:44.351605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-54382182-bb9f-4090-98cb-32fad989b2e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ae28a715af7496b981260452de90546'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:44.381115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 932/1867] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:00:44 2022] DELETE /identity/v3/users/6ae28a715af7496b981260452de90546 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:44.382262 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54382182-bb9f-4090-98cb-32fad989b2e7 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:44.393735 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-54382182-bb9f-4090-98cb-32fad989b2e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:44.467031 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 936/1868] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:44.480969 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a4a30-344f-4b60-8299-698ac424aae1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:44.483294 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-708a4a30-344f-4b60-8299-698ac424aae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:44.497287 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:44.501717 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54382182-bb9f-4090-98cb-32fad989b2e7 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:44.513897 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5c21b-b03f-48e2-9e89-405eb35313cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:44.514668 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5c21b-b03f-48e2-9e89-405eb35313cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:44.515081 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5c21b-b03f-48e2-9e89-405eb35313cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:44.532368 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5c21b-b03f-48e2-9e89-405eb35313cd admin admin] PATH_INFO: `/v3/projects/1baef349b6654ebd9349c36769519192` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:44.538208 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5c21b-b03f-48e2-9e89-405eb35313cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1baef349b6654ebd9349c36769519192)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:44.540919 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5c21b-b03f-48e2-9e89-405eb35313cd admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:44.627790 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a4a30-344f-4b60-8299-698ac424aae1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:44.670987 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9ed5c21b-b03f-48e2-9e89-405eb35313cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1baef349b6654ebd9349c36769519192', 'request_id': 'req-9ed5c21b-b03f-48e2-9e89-405eb35313cd'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:44.689502 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9ed5c21b-b03f-48e2-9e89-405eb35313cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1baef349b6654ebd9349c36769519192', 'request_id': 'req-9ed5c21b-b03f-48e2-9e89-405eb35313cd'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:44.720267 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9ed5c21b-b03f-48e2-9e89-405eb35313cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1baef349b6654ebd9349c36769519192', 'request_id': 'req-9ed5c21b-b03f-48e2-9e89-405eb35313cd'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:44.759063 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ad660-dd08-456b-ad9d-e56972223968 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:44.759657 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9ed5c21b-b03f-48e2-9e89-405eb35313cd admin admin] The token cache is being invalidate because project 1baef349b6654ebd9349c36769519192 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:00:44.760108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9ed5c21b-b03f-48e2-9e89-405eb35313cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:44.760942 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ad660-dd08-456b-ad9d-e56972223968 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:44.761439 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ad660-dd08-456b-ad9d-e56972223968 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:44.761627 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9ed5c21b-b03f-48e2-9e89-405eb35313cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:44.762562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ad660-dd08-456b-ad9d-e56972223968 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:44.764480 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 933/1869] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:00:44 2022] DELETE /identity/v3/projects/1baef349b6654ebd9349c36769519192 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:44.770198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5c21b-b03f-48e2-9e89-405eb35313cd admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:44.772617 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-9ed5c21b-b03f-48e2-9e89-405eb35313cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:44.774460 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ea9ad660-dd08-456b-ad9d-e56972223968 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:44.831091 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ad660-dd08-456b-ad9d-e56972223968 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:44.833049 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ad660-dd08-456b-ad9d-e56972223968 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:44.850677 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 937/1870] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:00:44.890579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5c21b-b03f-48e2-9e89-405eb35313cd admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:44.898334 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a72d25f-ea97-4156-b1da-833aa520c487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:44.899338 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72d25f-ea97-4156-b1da-833aa520c487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:44.900528 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72d25f-ea97-4156-b1da-833aa520c487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:44.900528 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72d25f-ea97-4156-b1da-833aa520c487 admin admin] PATH_INFO: `/v3/users/53a70dda6772429ca3f1c0a0a2ac67b2` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:44.902640 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a72d25f-ea97-4156-b1da-833aa520c487 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53a70dda6772429ca3f1c0a0a2ac67b2)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:44.905797 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a72d25f-ea97-4156-b1da-833aa520c487 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:44.992991 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9a72d25f-ea97-4156-b1da-833aa520c487 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53a70dda6772429ca3f1c0a0a2ac67b2'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:45.005160 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9a72d25f-ea97-4156-b1da-833aa520c487 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53a70dda6772429ca3f1c0a0a2ac67b2'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:45.023034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9a72d25f-ea97-4156-b1da-833aa520c487 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53a70dda6772429ca3f1c0a0a2ac67b2'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:45.024490 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9a72d25f-ea97-4156-b1da-833aa520c487 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53a70dda6772429ca3f1c0a0a2ac67b2'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:45.028302 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9a72d25f-ea97-4156-b1da-833aa520c487 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53a70dda6772429ca3f1c0a0a2ac67b2'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:45.068616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 934/1871] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:00:44 2022] DELETE /identity/v3/users/53a70dda6772429ca3f1c0a0a2ac67b2 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:00:45.085796 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ad660-dd08-456b-ad9d-e56972223968 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:45.088494 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ea9ad660-dd08-456b-ad9d-e56972223968 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:45.109378 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ad660-dd08-456b-ad9d-e56972223968 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:45.239186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:45.269185 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b537d-007f-4105-8887-bf3fed512ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:45.269947 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b537d-007f-4105-8887-bf3fed512ba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:45.270232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b537d-007f-4105-8887-bf3fed512ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:45.270509 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b537d-007f-4105-8887-bf3fed512ba6 admin admin] PATH_INFO: `/v3/projects/145b5b1ebb8640fe80a2e52e766b2a69` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:45.272329 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4b537d-007f-4105-8887-bf3fed512ba6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=145b5b1ebb8640fe80a2e52e766b2a69)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:45.273781 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4b537d-007f-4105-8887-bf3fed512ba6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:45.348644 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a72d25f-ea97-4156-b1da-833aa520c487 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:45.350512 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-9a72d25f-ea97-4156-b1da-833aa520c487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:45.381490 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a72d25f-ea97-4156-b1da-833aa520c487 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:45.390019 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:45.436093 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9f4b537d-007f-4105-8887-bf3fed512ba6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '145b5b1ebb8640fe80a2e52e766b2a69', 'request_id': 'req-9f4b537d-007f-4105-8887-bf3fed512ba6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:45.447115 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9f4b537d-007f-4105-8887-bf3fed512ba6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '145b5b1ebb8640fe80a2e52e766b2a69', 'request_id': 'req-9f4b537d-007f-4105-8887-bf3fed512ba6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:45.473434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9f4b537d-007f-4105-8887-bf3fed512ba6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '145b5b1ebb8640fe80a2e52e766b2a69', 'request_id': 'req-9f4b537d-007f-4105-8887-bf3fed512ba6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:45.500437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9f4b537d-007f-4105-8887-bf3fed512ba6 admin admin] The token cache is being invalidate because project 145b5b1ebb8640fe80a2e52e766b2a69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:00:45.501072 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9f4b537d-007f-4105-8887-bf3fed512ba6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:45.502365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9f4b537d-007f-4105-8887-bf3fed512ba6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:45.505381 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 938/1872] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:00:45 2022] DELETE /identity/v3/projects/145b5b1ebb8640fe80a2e52e766b2a69 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:45.512639 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b3c00-890c-4e3d-9aa1-031deecae5e3 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:45.513415 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b3c00-890c-4e3d-9aa1-031deecae5e3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:45.513978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b3c00-890c-4e3d-9aa1-031deecae5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:45.555379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-469b3c00-890c-4e3d-9aa1-031deecae5e3 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:45.555826 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35337e3b-0e09-4728-9b35-cf5ad48f4e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:45.557481 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35337e3b-0e09-4728-9b35-cf5ad48f4e8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:45.558358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35337e3b-0e09-4728-9b35-cf5ad48f4e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:45.559055 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35337e3b-0e09-4728-9b35-cf5ad48f4e8b admin admin] PATH_INFO: `/v3/projects/a3acdb67fcff463695c57f69b1e5a54d` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:45.559250 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-469b3c00-890c-4e3d-9aa1-031deecae5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:45.561655 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35337e3b-0e09-4728-9b35-cf5ad48f4e8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3acdb67fcff463695c57f69b1e5a54d)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:45.566704 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35337e3b-0e09-4728-9b35-cf5ad48f4e8b admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:45.573241 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 939/1873] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:45.590292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860edca-9dd0-41f7-ae1a-df2706961ab7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:45.590691 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860edca-9dd0-41f7-ae1a-df2706961ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:45.591841 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860edca-9dd0-41f7-ae1a-df2706961ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:45.631425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7860edca-9dd0-41f7-ae1a-df2706961ab7 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:45.633467 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7860edca-9dd0-41f7-ae1a-df2706961ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:45.656364 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 940/1874] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:00:45.666666 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860edca-9dd0-41f7-ae1a-df2706961ab7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:45.684807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860edca-9dd0-41f7-ae1a-df2706961ab7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:45.702407 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-35337e3b-0e09-4728-9b35-cf5ad48f4e8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3acdb67fcff463695c57f69b1e5a54d', 'request_id': 'req-35337e3b-0e09-4728-9b35-cf5ad48f4e8b'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:45.713774 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-35337e3b-0e09-4728-9b35-cf5ad48f4e8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3acdb67fcff463695c57f69b1e5a54d', 'request_id': 'req-35337e3b-0e09-4728-9b35-cf5ad48f4e8b'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:45.723740 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-35337e3b-0e09-4728-9b35-cf5ad48f4e8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3acdb67fcff463695c57f69b1e5a54d', 'request_id': 'req-35337e3b-0e09-4728-9b35-cf5ad48f4e8b'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:45.740946 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-35337e3b-0e09-4728-9b35-cf5ad48f4e8b admin admin] The token cache is being invalidate because project a3acdb67fcff463695c57f69b1e5a54d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:00:45.741224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-35337e3b-0e09-4728-9b35-cf5ad48f4e8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:45.741966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-35337e3b-0e09-4728-9b35-cf5ad48f4e8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:45.744627 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 935/1875] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:00:45 2022] DELETE /identity/v3/projects/a3acdb67fcff463695c57f69b1e5a54d => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:00:45.749193 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35337e3b-0e09-4728-9b35-cf5ad48f4e8b admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:45.751142 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-35337e3b-0e09-4728-9b35-cf5ad48f4e8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:45.774594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3794aa-c987-40b3-a310-269340018fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:45.775352 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3794aa-c987-40b3-a310-269340018fce admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:45.775674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3794aa-c987-40b3-a310-269340018fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:45.775953 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3794aa-c987-40b3-a310-269340018fce admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:45.776842 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3794aa-c987-40b3-a310-269340018fce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:00:45.777301 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3794aa-c987-40b3-a310-269340018fce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:45.779442 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3794aa-c987-40b3-a310-269340018fce admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:45.795056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 941/1876] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:00:45.799624 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3794aa-c987-40b3-a310-269340018fce admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:45.801692 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0d3794aa-c987-40b3-a310-269340018fce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:45.819597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3794aa-c987-40b3-a310-269340018fce admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:45.823831 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371ded5-fdf0-40f2-8808-07930c3ad878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:45.824479 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371ded5-fdf0-40f2-8808-07930c3ad878 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:45.824757 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371ded5-fdf0-40f2-8808-07930c3ad878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:45.825033 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371ded5-fdf0-40f2-8808-07930c3ad878 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:45.825882 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371ded5-fdf0-40f2-8808-07930c3ad878 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:45.828267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371ded5-fdf0-40f2-8808-07930c3ad878 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:45.840241 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35337e3b-0e09-4728-9b35-cf5ad48f4e8b admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:45.845709 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de6211-36c7-4e3b-8bb4-1d5e9ebec799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:45.846431 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de6211-36c7-4e3b-8bb4-1d5e9ebec799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:45.846788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de6211-36c7-4e3b-8bb4-1d5e9ebec799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:45.846948 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de6211-36c7-4e3b-8bb4-1d5e9ebec799 admin admin] PATH_INFO: `/v3/users/e5dbc643b386409985631ef6ecbb7bdf` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:45.848341 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de6211-36c7-4e3b-8bb4-1d5e9ebec799 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5dbc643b386409985631ef6ecbb7bdf)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:45.850314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de6211-36c7-4e3b-8bb4-1d5e9ebec799 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:45.893857 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 942/1877] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:00:45 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:00:45.899537 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371ded5-fdf0-40f2-8808-07930c3ad878 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:45.932593 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:45.952253 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-55de6211-36c7-4e3b-8bb4-1d5e9ebec799 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5dbc643b386409985631ef6ecbb7bdf'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:45.966661 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-55de6211-36c7-4e3b-8bb4-1d5e9ebec799 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5dbc643b386409985631ef6ecbb7bdf'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:45.978042 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:45.988857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-55de6211-36c7-4e3b-8bb4-1d5e9ebec799 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5dbc643b386409985631ef6ecbb7bdf'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:45.989659 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-55de6211-36c7-4e3b-8bb4-1d5e9ebec799 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5dbc643b386409985631ef6ecbb7bdf'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:45.990308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-55de6211-36c7-4e3b-8bb4-1d5e9ebec799 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5dbc643b386409985631ef6ecbb7bdf'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:46.027920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 936/1878] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:00:45 2022] DELETE /identity/v3/users/e5dbc643b386409985631ef6ecbb7bdf => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:00:46.028906 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371ded5-fdf0-40f2-8808-07930c3ad878 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:46.119936 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d72a96-bef6-4876-a987-e3231e00eb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:46.120633 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d72a96-bef6-4876-a987-e3231e00eb4e admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:46.121278 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d72a96-bef6-4876-a987-e3231e00eb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:46.121472 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d72a96-bef6-4876-a987-e3231e00eb4e admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:46.122897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d72a96-bef6-4876-a987-e3231e00eb4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:46.125292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d72a96-bef6-4876-a987-e3231e00eb4e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:46.168856 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de6211-36c7-4e3b-8bb4-1d5e9ebec799 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:46.169652 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 943/1879] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:00:45 2022] POST /identity/v3/users => generated 292 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:46.170457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-55de6211-36c7-4e3b-8bb4-1d5e9ebec799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:46.198100 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de6211-36c7-4e3b-8bb4-1d5e9ebec799 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:46.209933 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a7a898-72ec-435c-8174-757332564a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:46.210683 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7a898-72ec-435c-8174-757332564a4d admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:46.210803 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7a898-72ec-435c-8174-757332564a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:46.211373 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7a898-72ec-435c-8174-757332564a4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:46.211804 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a7a898-72ec-435c-8174-757332564a4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:46.213552 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a7a898-72ec-435c-8174-757332564a4d admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:46.226377 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 937/1880] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:00:46.231620 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d72a96-bef6-4876-a987-e3231e00eb4e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:46.254942 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d72a96-bef6-4876-a987-e3231e00eb4e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:46.260392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10ca76-7c43-44be-9683-a193318f243f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:46.261720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10ca76-7c43-44be-9683-a193318f243f admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:46.262811 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10ca76-7c43-44be-9683-a193318f243f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:46.263267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10ca76-7c43-44be-9683-a193318f243f admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:46.264128 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10ca76-7c43-44be-9683-a193318f243f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:46.268134 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10ca76-7c43-44be-9683-a193318f243f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:46.280223 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a7a898-72ec-435c-8174-757332564a4d admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:46.282553 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-87a7a898-72ec-435c-8174-757332564a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:46.296704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 944/1881] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:46.301798 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10ca76-7c43-44be-9683-a193318f243f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:46.310897 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a7a898-72ec-435c-8174-757332564a4d admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:46.320668 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899a5c2-de4b-4fb0-8516-5cd2468b01cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:46.321702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899a5c2-de4b-4fb0-8516-5cd2468b01cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:46.323494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899a5c2-de4b-4fb0-8516-5cd2468b01cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:46.323973 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899a5c2-de4b-4fb0-8516-5cd2468b01cc admin admin] PATH_INFO: `/v3/projects/4e4e4cd47e784a81a599cae89c51caaa` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:46.326722 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899a5c2-de4b-4fb0-8516-5cd2468b01cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e4e4cd47e784a81a599cae89c51caaa)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:46.330697 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899a5c2-de4b-4fb0-8516-5cd2468b01cc admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:46.357103 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10ca76-7c43-44be-9683-a193318f243f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:46.375876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7456dc-17c3-417f-bb27-326a82f5dd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:46.376591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7456dc-17c3-417f-bb27-326a82f5dd56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:46.376878 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7456dc-17c3-417f-bb27-326a82f5dd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:46.377159 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7456dc-17c3-417f-bb27-326a82f5dd56 admin admin] PATH_INFO: `/v3/projects/22f6a027b2074b41ba4ac9f5f513070a/users/12396b0ad6604ae2873388175dc24ad3/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:46.397688 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7456dc-17c3-417f-bb27-326a82f5dd56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22f6a027b2074b41ba4ac9f5f513070a, user_id=12396b0ad6604ae2873388175dc24ad3, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:46.400483 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7456dc-17c3-417f-bb27-326a82f5dd56 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:46.430527 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9899a5c2-de4b-4fb0-8516-5cd2468b01cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e4e4cd47e784a81a599cae89c51caaa', 'request_id': 'req-9899a5c2-de4b-4fb0-8516-5cd2468b01cc'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:46.436844 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 945/1882] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:46 2022] PUT /identity/v3/projects/22f6a027b2074b41ba4ac9f5f513070a/users/12396b0ad6604ae2873388175dc24ad3/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:00:46.448354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2befe6-4708-451d-b445-f6e039d36c61 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:46.448653 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2befe6-4708-451d-b445-f6e039d36c61 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:46.448935 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2befe6-4708-451d-b445-f6e039d36c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:46.453573 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9899a5c2-de4b-4fb0-8516-5cd2468b01cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e4e4cd47e784a81a599cae89c51caaa', 'request_id': 'req-9899a5c2-de4b-4fb0-8516-5cd2468b01cc'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:46.466186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9899a5c2-de4b-4fb0-8516-5cd2468b01cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e4e4cd47e784a81a599cae89c51caaa', 'request_id': 'req-9899a5c2-de4b-4fb0-8516-5cd2468b01cc'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:46.491614 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9899a5c2-de4b-4fb0-8516-5cd2468b01cc admin admin] The token cache is being invalidate because project 4e4e4cd47e784a81a599cae89c51caaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:00:46.491921 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9899a5c2-de4b-4fb0-8516-5cd2468b01cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:46.500227 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9899a5c2-de4b-4fb0-8516-5cd2468b01cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:46.506756 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 938/1883] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:00:46 2022] DELETE /identity/v3/projects/4e4e4cd47e784a81a599cae89c51caaa => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:46.513363 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22b3d4-524f-4324-bad1-028fb8c01c8e None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:46.513718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22b3d4-524f-4324-bad1-028fb8c01c8e None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:46.513941 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22b3d4-524f-4324-bad1-028fb8c01c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:46.524693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3e2befe6-4708-451d-b445-f6e039d36c61 None None] MFA Rules not processed for user `12396b0ad6604ae2873388175dc24ad3`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:46.526314 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3e2befe6-4708-451d-b445-f6e039d36c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:46.550644 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-8c22b3d4-524f-4324-bad1-028fb8c01c8e None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:46.552472 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-8c22b3d4-524f-4324-bad1-028fb8c01c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:46.627370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 946/1884] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:00:46.640829 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88990e35-8f01-42d3-88fa-aa0f19949aeb None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:46.641118 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88990e35-8f01-42d3-88fa-aa0f19949aeb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:46.642829 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88990e35-8f01-42d3-88fa-aa0f19949aeb None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:46.644944 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 947/1885] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jul 26 05:00:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 05:00:46.646055 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88990e35-8f01-42d3-88fa-aa0f19949aeb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:46.650872 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-88990e35-8f01-42d3-88fa-aa0f19949aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:46.665024 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 939/1886] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:46.676147 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86493b0-2915-4ec3-a0a6-206c78051cb7 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:46.676834 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86493b0-2915-4ec3-a0a6-206c78051cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:46.679528 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86493b0-2915-4ec3-a0a6-206c78051cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:46.717589 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:46.723858 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-c86493b0-2915-4ec3-a0a6-206c78051cb7 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:46.725707 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c86493b0-2915-4ec3-a0a6-206c78051cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:46.743314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 940/1887] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:46.745207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c42a6-13d8-4258-bae9-a2d8d603c7f9 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:46.745654 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c42a6-13d8-4258-bae9-a2d8d603c7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:46.746195 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c42a6-13d8-4258-bae9-a2d8d603c7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:46.770496 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-e54c42a6-13d8-4258-bae9-a2d8d603c7f9 None None] MFA Rules not processed for user `2faeed9002ee4154bc980c146cd06bcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:46.771805 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-e54c42a6-13d8-4258-bae9-a2d8d603c7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:46.774354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88990e35-8f01-42d3-88fa-aa0f19949aeb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:46.778935 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364b387-44e3-4c82-a882-bf065c007616 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:46.780047 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364b387-44e3-4c82-a882-bf065c007616 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:46.780537 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364b387-44e3-4c82-a882-bf065c007616 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:46.780915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364b387-44e3-4c82-a882-bf065c007616 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:46.794578 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364b387-44e3-4c82-a882-bf065c007616 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:46.796388 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364b387-44e3-4c82-a882-bf065c007616 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:46.804727 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:46.810649 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 948/1888] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:46.818385 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364b387-44e3-4c82-a882-bf065c007616 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:46.821437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0364b387-44e3-4c82-a882-bf065c007616 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:46.840878 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364b387-44e3-4c82-a882-bf065c007616 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:46.845779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bda620-e0ea-472c-95f9-d0a43035afd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:46.846652 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bda620-e0ea-472c-95f9-d0a43035afd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:46.847216 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bda620-e0ea-472c-95f9-d0a43035afd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:46.847390 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bda620-e0ea-472c-95f9-d0a43035afd5 admin admin] PATH_INFO: `/v3/projects/f8098c6aff364cb89a5f77a55a3a1029` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:46.848858 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bda620-e0ea-472c-95f9-d0a43035afd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8098c6aff364cb89a5f77a55a3a1029)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:46.850960 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bda620-e0ea-472c-95f9-d0a43035afd5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:46.879006 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 941/1889] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jul 26 05:00:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:46.879967 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c42a6-13d8-4258-bae9-a2d8d603c7f9 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:46.893250 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c42a6-13d8-4258-bae9-a2d8d603c7f9 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:46.902757 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab16aa-c021-4911-bdfd-1232712cbd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:46.903618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab16aa-c021-4911-bdfd-1232712cbd77 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:46.904015 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab16aa-c021-4911-bdfd-1232712cbd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:46.904627 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab16aa-c021-4911-bdfd-1232712cbd77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:46.905651 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ab16aa-c021-4911-bdfd-1232712cbd77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:00:46.905955 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ab16aa-c021-4911-bdfd-1232712cbd77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:46.907652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ab16aa-c021-4911-bdfd-1232712cbd77 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:46.928741 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 942/1890] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:00:46.930294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab16aa-c021-4911-bdfd-1232712cbd77 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:46.930833 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-36bda620-e0ea-472c-95f9-d0a43035afd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8098c6aff364cb89a5f77a55a3a1029', 'request_id': 'req-36bda620-e0ea-472c-95f9-d0a43035afd5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:46.932123 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a0ab16aa-c021-4911-bdfd-1232712cbd77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:46.938671 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-36bda620-e0ea-472c-95f9-d0a43035afd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8098c6aff364cb89a5f77a55a3a1029', 'request_id': 'req-36bda620-e0ea-472c-95f9-d0a43035afd5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:46.949074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab16aa-c021-4911-bdfd-1232712cbd77 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:46.949357 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-36bda620-e0ea-472c-95f9-d0a43035afd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8098c6aff364cb89a5f77a55a3a1029', 'request_id': 'req-36bda620-e0ea-472c-95f9-d0a43035afd5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:46.966769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-36bda620-e0ea-472c-95f9-d0a43035afd5 admin admin] The token cache is being invalidate because project f8098c6aff364cb89a5f77a55a3a1029 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:00:46.967266 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-36bda620-e0ea-472c-95f9-d0a43035afd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:46.968371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-36bda620-e0ea-472c-95f9-d0a43035afd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:46.970535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 949/1891] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:00:46 2022] DELETE /identity/v3/projects/f8098c6aff364cb89a5f77a55a3a1029 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:46.971398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bda620-e0ea-472c-95f9-d0a43035afd5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:46.973111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-36bda620-e0ea-472c-95f9-d0a43035afd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:47.051578 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9567a1-2387-4b51-8898-dff80e5b5a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:47.052810 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9567a1-2387-4b51-8898-dff80e5b5a1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:47.053333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9567a1-2387-4b51-8898-dff80e5b5a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:47.053779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9567a1-2387-4b51-8898-dff80e5b5a1f admin admin] PATH_INFO: `/v3/users/12396b0ad6604ae2873388175dc24ad3` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:47.055755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9567a1-2387-4b51-8898-dff80e5b5a1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=12396b0ad6604ae2873388175dc24ad3)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:47.059410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9567a1-2387-4b51-8898-dff80e5b5a1f admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:47.069416 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bda620-e0ea-472c-95f9-d0a43035afd5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:47.074433 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449257f-6fd0-4ef1-b184-bf00cb6d89e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:47.075078 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449257f-6fd0-4ef1-b184-bf00cb6d89e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:47.075359 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449257f-6fd0-4ef1-b184-bf00cb6d89e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:47.075666 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449257f-6fd0-4ef1-b184-bf00cb6d89e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:47.076866 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449257f-6fd0-4ef1-b184-bf00cb6d89e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:47.078838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449257f-6fd0-4ef1-b184-bf00cb6d89e3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:47.119163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 950/1892] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:00:46 2022] POST /identity/v3/projects => generated 387 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:47.122220 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb83232-5008-4308-80f2-692a8d428b43 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:47.123734 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb83232-5008-4308-80f2-692a8d428b43 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:47.125177 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb83232-5008-4308-80f2-692a8d428b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:47.142592 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-2a9567a1-2387-4b51-8898-dff80e5b5a1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12396b0ad6604ae2873388175dc24ad3'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:47.153192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-2a9567a1-2387-4b51-8898-dff80e5b5a1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12396b0ad6604ae2873388175dc24ad3'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:47.156518 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1bb83232-5008-4308-80f2-692a8d428b43 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:47.158276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1bb83232-5008-4308-80f2-692a8d428b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:47.164392 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-2a9567a1-2387-4b51-8898-dff80e5b5a1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12396b0ad6604ae2873388175dc24ad3'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:47.165611 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-2a9567a1-2387-4b51-8898-dff80e5b5a1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12396b0ad6604ae2873388175dc24ad3'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:47.166316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-2a9567a1-2387-4b51-8898-dff80e5b5a1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12396b0ad6604ae2873388175dc24ad3'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:47.192067 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 943/1893] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:00:46 2022] DELETE /identity/v3/users/12396b0ad6604ae2873388175dc24ad3 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:47.195966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9567a1-2387-4b51-8898-dff80e5b5a1f admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:47.197660 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-2a9567a1-2387-4b51-8898-dff80e5b5a1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:47.212507 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:47.227734 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:47.294249 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 951/1894] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:47.294249 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb83232-5008-4308-80f2-692a8d428b43 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:47.294249 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1bb83232-5008-4308-80f2-692a8d428b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:47.299389 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9567a1-2387-4b51-8898-dff80e5b5a1f admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:47.305568 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f444cc-9e70-4507-9b22-a632a1c4e1ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:47.306296 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f444cc-9e70-4507-9b22-a632a1c4e1ee service nova] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:47.306637 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f444cc-9e70-4507-9b22-a632a1c4e1ee service nova] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:47.307061 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f444cc-9e70-4507-9b22-a632a1c4e1ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:47.309065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-83f444cc-9e70-4507-9b22-a632a1c4e1ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:47.311356 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb83232-5008-4308-80f2-692a8d428b43 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:47.322459 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f444cc-9e70-4507-9b22-a632a1c4e1ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:47.324300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f444cc-9e70-4507-9b22-a632a1c4e1ee service nova] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:47.339183 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 944/1895] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:47.339183 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953cdf00-58fc-462a-a5de-444b03af2716 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:47.339183 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953cdf00-58fc-462a-a5de-444b03af2716 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:47.339183 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953cdf00-58fc-462a-a5de-444b03af2716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:47.368933 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-953cdf00-58fc-462a-a5de-444b03af2716 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:47.374010 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-953cdf00-58fc-462a-a5de-444b03af2716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:47.477721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c56fd85-9a02-4f36-ac3b-0d88a6b8b642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:47.478383 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56fd85-9a02-4f36-ac3b-0d88a6b8b642 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:47.478693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56fd85-9a02-4f36-ac3b-0d88a6b8b642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:47.479010 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56fd85-9a02-4f36-ac3b-0d88a6b8b642 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:47.479749 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c56fd85-9a02-4f36-ac3b-0d88a6b8b642 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:47.483260 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c56fd85-9a02-4f36-ac3b-0d88a6b8b642 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:47.518906 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 945/1896] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:47.518906 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbab668-a6ff-4869-b91d-c7219e39d127 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:47.518906 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbab668-a6ff-4869-b91d-c7219e39d127 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:47.518906 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbab668-a6ff-4869-b91d-c7219e39d127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:47.527740 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 952/1897] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:00:47 2022] POST /identity/v3/users => generated 294 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:47.531311 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c56fd85-9a02-4f36-ac3b-0d88a6b8b642 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:47.558517 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c56fd85-9a02-4f36-ac3b-0d88a6b8b642 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:47.564678 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b530f-25a6-4765-9288-a85871d1f7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:47.565080 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-cdbab668-a6ff-4869-b91d-c7219e39d127 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:47.565553 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b530f-25a6-4765-9288-a85871d1f7ff admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:47.565930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b530f-25a6-4765-9288-a85871d1f7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:47.566333 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b530f-25a6-4765-9288-a85871d1f7ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:47.566450 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-cdbab668-a6ff-4869-b91d-c7219e39d127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:47.567228 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b530f-25a6-4765-9288-a85871d1f7ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:47.569255 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b530f-25a6-4765-9288-a85871d1f7ff admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:47.584915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 953/1898] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:47.584915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b530f-25a6-4765-9288-a85871d1f7ff admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:47.585365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-991b530f-25a6-4765-9288-a85871d1f7ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:47.589920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 946/1899] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:47.594078 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbab668-a6ff-4869-b91d-c7219e39d127 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:47.609172 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbab668-a6ff-4869-b91d-c7219e39d127 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:47.616291 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73def8-903b-48a5-a7cd-840525072779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:47.617471 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73def8-903b-48a5-a7cd-840525072779 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:47.618064 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73def8-903b-48a5-a7cd-840525072779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:47.618519 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73def8-903b-48a5-a7cd-840525072779 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:47.619667 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee73def8-903b-48a5-a7cd-840525072779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:47.621935 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee73def8-903b-48a5-a7cd-840525072779 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:47.639587 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 947/1900] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:47.640565 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73def8-903b-48a5-a7cd-840525072779 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:47.664486 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73def8-903b-48a5-a7cd-840525072779 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:47.669188 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b530f-25a6-4765-9288-a85871d1f7ff admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:47.673096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a18bf0-3495-4c78-be10-d8c3c80706ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:47.673861 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe996526-7dd3-4844-b2b1-be9265aa6ea5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:47.674399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a18bf0-3495-4c78-be10-d8c3c80706ed admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:47.674909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a18bf0-3495-4c78-be10-d8c3c80706ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:47.675491 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a18bf0-3495-4c78-be10-d8c3c80706ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:47.676586 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a18bf0-3495-4c78-be10-d8c3c80706ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:00:47.677334 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a18bf0-3495-4c78-be10-d8c3c80706ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:47.679245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a18bf0-3495-4c78-be10-d8c3c80706ed admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:47.684096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe996526-7dd3-4844-b2b1-be9265aa6ea5 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:47.684448 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe996526-7dd3-4844-b2b1-be9265aa6ea5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:47.692667 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe996526-7dd3-4844-b2b1-be9265aa6ea5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:47.699761 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 948/1901] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:00:47.700908 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a18bf0-3495-4c78-be10-d8c3c80706ed admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:47.708395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe996526-7dd3-4844-b2b1-be9265aa6ea5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:47.709864 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe996526-7dd3-4844-b2b1-be9265aa6ea5 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:47.722763 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 954/1902] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:47.724403 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe996526-7dd3-4844-b2b1-be9265aa6ea5 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:47.728440 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a18bf0-3495-4c78-be10-d8c3c80706ed admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:47.733843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255cdc8c-c125-42f0-8769-07d711646c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:47.734808 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255cdc8c-c125-42f0-8769-07d711646c99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:47.735280 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255cdc8c-c125-42f0-8769-07d711646c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:47.735655 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255cdc8c-c125-42f0-8769-07d711646c99 admin admin] PATH_INFO: `/v3/projects/da7821e3dcbb4ddb9e9ed57e171dee6f/users/8aeff3e2c3dd45d487af606dfd2fa12d/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:47.737505 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe996526-7dd3-4844-b2b1-be9265aa6ea5 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:47.743737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58fb99-9fac-4531-bf15-46b40c8cff51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:47.744551 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58fb99-9fac-4531-bf15-46b40c8cff51 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:47.745068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58fb99-9fac-4531-bf15-46b40c8cff51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:47.745605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58fb99-9fac-4531-bf15-46b40c8cff51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:47.746434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e58fb99-9fac-4531-bf15-46b40c8cff51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:47.748926 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e58fb99-9fac-4531-bf15-46b40c8cff51 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:47.755676 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255cdc8c-c125-42f0-8769-07d711646c99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da7821e3dcbb4ddb9e9ed57e171dee6f, user_id=8aeff3e2c3dd45d487af606dfd2fa12d, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:47.758475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255cdc8c-c125-42f0-8769-07d711646c99 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:47.774963 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 949/1903] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:47 2022] PUT /identity/v3/projects/da7821e3dcbb4ddb9e9ed57e171dee6f/users/8aeff3e2c3dd45d487af606dfd2fa12d/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:47.787941 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d1109-76b0-4a78-ae37-2c42807e3763 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:47.788232 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d1109-76b0-4a78-ae37-2c42807e3763 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:47.788511 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d1109-76b0-4a78-ae37-2c42807e3763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:47.829496 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 955/1904] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:00:47 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:47.834008 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58fb99-9fac-4531-bf15-46b40c8cff51 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:47.836748 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-e65d1109-76b0-4a78-ae37-2c42807e3763 None None] MFA Rules not processed for user `8aeff3e2c3dd45d487af606dfd2fa12d`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:47.838428 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-e65d1109-76b0-4a78-ae37-2c42807e3763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:47.895034 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:47.906738 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:47.930821 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58fb99-9fac-4531-bf15-46b40c8cff51 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:47.938857 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bce035-8d3e-43a2-b028-19cf7bfc0a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:47.939891 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bce035-8d3e-43a2-b028-19cf7bfc0a8e admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:47.940492 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bce035-8d3e-43a2-b028-19cf7bfc0a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:47.940969 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bce035-8d3e-43a2-b028-19cf7bfc0a8e admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:47.942585 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bce035-8d3e-43a2-b028-19cf7bfc0a8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:47.944371 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 950/1905] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:00:47.947109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bce035-8d3e-43a2-b028-19cf7bfc0a8e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:47.948905 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d1109-76b0-4a78-ae37-2c42807e3763 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:47.968639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d1109-76b0-4a78-ae37-2c42807e3763 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:47.974646 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545c389-c2d6-41ab-befb-25f716cedb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:47.975370 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545c389-c2d6-41ab-befb-25f716cedb29 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:47.975972 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 956/1906] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:00:47 2022] POST /identity/v3/users => generated 289 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:00:47.978968 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545c389-c2d6-41ab-befb-25f716cedb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:47.979283 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545c389-c2d6-41ab-befb-25f716cedb29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:47.980134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8545c389-c2d6-41ab-befb-25f716cedb29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:47.980809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bce035-8d3e-43a2-b028-19cf7bfc0a8e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:47.982283 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8545c389-c2d6-41ab-befb-25f716cedb29 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:48.003187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bce035-8d3e-43a2-b028-19cf7bfc0a8e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:48.009549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48087e42-eeae-42fa-8a4f-2a9b9b7304af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:48.010381 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48087e42-eeae-42fa-8a4f-2a9b9b7304af admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:48.010822 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48087e42-eeae-42fa-8a4f-2a9b9b7304af admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:48.011257 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48087e42-eeae-42fa-8a4f-2a9b9b7304af admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:48.012144 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48087e42-eeae-42fa-8a4f-2a9b9b7304af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:48.013931 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48087e42-eeae-42fa-8a4f-2a9b9b7304af admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:48.022981 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 951/1907] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:00:47 2022] POST /identity/v3/projects => generated 389 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:48.024189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545c389-c2d6-41ab-befb-25f716cedb29 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:48.026981 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 957/1908] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:48.031135 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48087e42-eeae-42fa-8a4f-2a9b9b7304af admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:48.042718 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48087e42-eeae-42fa-8a4f-2a9b9b7304af admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:48.145354 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545c389-c2d6-41ab-befb-25f716cedb29 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:48.152115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099af401-2b88-488c-8274-3864187a48a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:48.152430 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099af401-2b88-488c-8274-3864187a48a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:48.152719 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099af401-2b88-488c-8274-3864187a48a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:48.153075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099af401-2b88-488c-8274-3864187a48a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:48.153679 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099af401-2b88-488c-8274-3864187a48a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:48.171036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099af401-2b88-488c-8274-3864187a48a5 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:48.171250 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6df40e-4b6d-4670-a115-132bfdd0aa84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:48.172019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6df40e-4b6d-4670-a115-132bfdd0aa84 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:48.172316 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6df40e-4b6d-4670-a115-132bfdd0aa84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:48.172596 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6df40e-4b6d-4670-a115-132bfdd0aa84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:48.174670 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6df40e-4b6d-4670-a115-132bfdd0aa84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:48.184490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6df40e-4b6d-4670-a115-132bfdd0aa84 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:48.215853 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 958/1909] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:48.217384 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6df40e-4b6d-4670-a115-132bfdd0aa84 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:48.239249 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 952/1910] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:00:48 2022] POST /identity/v3/users => generated 301 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:48.242981 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099af401-2b88-488c-8274-3864187a48a5 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:48.245606 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2d6df40e-4b6d-4670-a115-132bfdd0aa84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:48.256476 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099af401-2b88-488c-8274-3864187a48a5 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:48.261963 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf9cf7-3a10-4cc6-bf3f-ddfd4541f2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:48.262659 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf9cf7-3a10-4cc6-bf3f-ddfd4541f2d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:48.262995 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf9cf7-3a10-4cc6-bf3f-ddfd4541f2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:48.263275 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf9cf7-3a10-4cc6-bf3f-ddfd4541f2d2 admin admin] PATH_INFO: `/v3/projects/bf3d2754f23e496bb7b2f2531297380d/users/3a37ce859cd541878c8071a85b50eab9/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:48.279891 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6df40e-4b6d-4670-a115-132bfdd0aa84 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:48.283351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf9cf7-3a10-4cc6-bf3f-ddfd4541f2d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf3d2754f23e496bb7b2f2531297380d, user_id=3a37ce859cd541878c8071a85b50eab9, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:48.286270 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf9cf7-3a10-4cc6-bf3f-ddfd4541f2d2 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:48.298029 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efc11a-573c-4cc1-8005-7361073385b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:48.301165 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efc11a-573c-4cc1-8005-7361073385b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:48.304041 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efc11a-573c-4cc1-8005-7361073385b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:48.304041 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efc11a-573c-4cc1-8005-7361073385b1 admin admin] PATH_INFO: `/v3/projects/22f6a027b2074b41ba4ac9f5f513070a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:48.308736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3efc11a-573c-4cc1-8005-7361073385b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22f6a027b2074b41ba4ac9f5f513070a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:48.315855 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3efc11a-573c-4cc1-8005-7361073385b1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:48.349846 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 953/1911] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:48 2022] PUT /identity/v3/projects/bf3d2754f23e496bb7b2f2531297380d/users/3a37ce859cd541878c8071a85b50eab9/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:48.351363 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf9cf7-3a10-4cc6-bf3f-ddfd4541f2d2 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:48.459658 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c3efc11a-573c-4cc1-8005-7361073385b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22f6a027b2074b41ba4ac9f5f513070a', 'request_id': 'req-c3efc11a-573c-4cc1-8005-7361073385b1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:48.471084 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c3efc11a-573c-4cc1-8005-7361073385b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22f6a027b2074b41ba4ac9f5f513070a', 'request_id': 'req-c3efc11a-573c-4cc1-8005-7361073385b1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:48.486120 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c3efc11a-573c-4cc1-8005-7361073385b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22f6a027b2074b41ba4ac9f5f513070a', 'request_id': 'req-c3efc11a-573c-4cc1-8005-7361073385b1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:48.506302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c3efc11a-573c-4cc1-8005-7361073385b1 admin admin] The token cache is being invalidate because project 22f6a027b2074b41ba4ac9f5f513070a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:00:48.506664 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c3efc11a-573c-4cc1-8005-7361073385b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:48.507417 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c3efc11a-573c-4cc1-8005-7361073385b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:48.516256 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 959/1912] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:00:48 2022] DELETE /identity/v3/projects/22f6a027b2074b41ba4ac9f5f513070a => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:48.518059 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1b193-b8cc-4b85-a49a-3a5ccdfc1d0c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:48.518803 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1b193-b8cc-4b85-a49a-3a5ccdfc1d0c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:48.519319 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1b193-b8cc-4b85-a49a-3a5ccdfc1d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:48.521204 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf9cf7-3a10-4cc6-bf3f-ddfd4541f2d2 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:48.567350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6ea1b193-b8cc-4b85-a49a-3a5ccdfc1d0c None None] MFA Rules not processed for user `3a37ce859cd541878c8071a85b50eab9`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:48.569637 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6ea1b193-b8cc-4b85-a49a-3a5ccdfc1d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:48.591923 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:48.612925 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:48.630567 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca86b7-e63f-4c0c-81f5-2b9cafaf3a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:48.631800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca86b7-e63f-4c0c-81f5-2b9cafaf3a99 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:48.632228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca86b7-e63f-4c0c-81f5-2b9cafaf3a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:48.632620 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca86b7-e63f-4c0c-81f5-2b9cafaf3a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:48.633514 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ca86b7-e63f-4c0c-81f5-2b9cafaf3a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:48.635641 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ca86b7-e63f-4c0c-81f5-2b9cafaf3a99 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:48.648140 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 954/1913] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:48.650929 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b528a42-38d8-4bbd-b9eb-75de3c3a0a42 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:48.651696 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b528a42-38d8-4bbd-b9eb-75de3c3a0a42 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:48.652080 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b528a42-38d8-4bbd-b9eb-75de3c3a0a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:48.689521 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-1b528a42-38d8-4bbd-b9eb-75de3c3a0a42 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:48.690903 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-1b528a42-38d8-4bbd-b9eb-75de3c3a0a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:48.700044 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 960/1914] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:48.706303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1b193-b8cc-4b85-a49a-3a5ccdfc1d0c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:48.708847 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6ea1b193-b8cc-4b85-a49a-3a5ccdfc1d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:48.717242 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 955/1915] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:48.718126 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b528a42-38d8-4bbd-b9eb-75de3c3a0a42 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:48.726664 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-1b528a42-38d8-4bbd-b9eb-75de3c3a0a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:48.738696 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1b193-b8cc-4b85-a49a-3a5ccdfc1d0c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:48.744326 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a7434-5154-41b1-b18b-b017f48e48e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:48.745269 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a7434-5154-41b1-b18b-b017f48e48e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:48.745680 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a7434-5154-41b1-b18b-b017f48e48e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:48.746553 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a7434-5154-41b1-b18b-b017f48e48e1 admin admin] PATH_INFO: `/v3/projects/5d87219713724001927cd14519889d8a/users/e35673ddf25f4d279f82940f47a69026/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:48.748988 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b528a42-38d8-4bbd-b9eb-75de3c3a0a42 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:48.755195 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7534f90-f6ae-4bd3-b5f9-d61e7e4af98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:48.755949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7534f90-f6ae-4bd3-b5f9-d61e7e4af98c admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:48.756299 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7534f90-f6ae-4bd3-b5f9-d61e7e4af98c admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:48.756639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7534f90-f6ae-4bd3-b5f9-d61e7e4af98c admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:48.758092 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7534f90-f6ae-4bd3-b5f9-d61e7e4af98c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:48.760326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7534f90-f6ae-4bd3-b5f9-d61e7e4af98c admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:48.780892 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a7434-5154-41b1-b18b-b017f48e48e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d87219713724001927cd14519889d8a, user_id=e35673ddf25f4d279f82940f47a69026, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:48.783809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a7434-5154-41b1-b18b-b017f48e48e1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:48.806402 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 961/1916] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:48 2022] PUT /identity/v3/projects/5d87219713724001927cd14519889d8a/users/e35673ddf25f4d279f82940f47a69026/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:48.807889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0874b02-5aa1-4f76-b02e-da6cddc3afac None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:48.808175 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0874b02-5aa1-4f76-b02e-da6cddc3afac None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:48.808447 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0874b02-5aa1-4f76-b02e-da6cddc3afac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:48.811701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 956/1917] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:00:48 2022] POST /identity/v3/projects => generated 377 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:48.811701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7534f90-f6ae-4bd3-b5f9-d61e7e4af98c admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:48.831283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c0874b02-5aa1-4f76-b02e-da6cddc3afac None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:48.832590 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c0874b02-5aa1-4f76-b02e-da6cddc3afac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:48.890392 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7534f90-f6ae-4bd3-b5f9-d61e7e4af98c admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:48.895652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3dabc-a589-403f-915c-3f1aa6240781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:48.896364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3dabc-a589-403f-915c-3f1aa6240781 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:48.896801 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3dabc-a589-403f-915c-3f1aa6240781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:48.897080 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3dabc-a589-403f-915c-3f1aa6240781 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:48.897805 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3dabc-a589-403f-915c-3f1aa6240781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:48.899515 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3dabc-a589-403f-915c-3f1aa6240781 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:48.911243 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 957/1918] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:48.912164 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3dabc-a589-403f-915c-3f1aa6240781 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:48.922824 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3dabc-a589-403f-915c-3f1aa6240781 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:48.932674 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e58cf4b-72cc-40da-a02f-ca729bdb4a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:48.933547 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58cf4b-72cc-40da-a02f-ca729bdb4a78 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:48.933801 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58cf4b-72cc-40da-a02f-ca729bdb4a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:48.934105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58cf4b-72cc-40da-a02f-ca729bdb4a78 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:48.934769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e58cf4b-72cc-40da-a02f-ca729bdb4a78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:48.937036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e58cf4b-72cc-40da-a02f-ca729bdb4a78 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:48.957302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 962/1919] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:48.957302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0874b02-5aa1-4f76-b02e-da6cddc3afac None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:48.968463 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0874b02-5aa1-4f76-b02e-da6cddc3afac None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:48.976182 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 958/1920] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:00:48 2022] POST /identity/v3/users => generated 295 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:48.976601 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb55e9-e5b0-4746-87c4-1c2c2a3ffd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:48.977792 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e58cf4b-72cc-40da-a02f-ca729bdb4a78 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:48.978769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb55e9-e5b0-4746-87c4-1c2c2a3ffd1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:48.979251 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb55e9-e5b0-4746-87c4-1c2c2a3ffd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:48.979651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb55e9-e5b0-4746-87c4-1c2c2a3ffd1b admin admin] PATH_INFO: `/v3/projects/5d87219713724001927cd14519889d8a/users/e35673ddf25f4d279f82940f47a69026/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:48.982046 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb55e9-e5b0-4746-87c4-1c2c2a3ffd1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d87219713724001927cd14519889d8a, user_id=e35673ddf25f4d279f82940f47a69026, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:48.984897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb55e9-e5b0-4746-87c4-1c2c2a3ffd1b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:48.994248 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e58cf4b-72cc-40da-a02f-ca729bdb4a78 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:48.999088 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 963/1921] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:48 2022] PUT /identity/v3/projects/5d87219713724001927cd14519889d8a/users/e35673ddf25f4d279f82940f47a69026/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:48.999883 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb55e9-e5b0-4746-87c4-1c2c2a3ffd1b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:49.110885 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72bb13-d792-4f8a-880e-19cc9d89892b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:49.111878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72bb13-d792-4f8a-880e-19cc9d89892b admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:49.112237 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72bb13-d792-4f8a-880e-19cc9d89892b admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:49.112755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72bb13-d792-4f8a-880e-19cc9d89892b admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:49.114025 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c72bb13-d792-4f8a-880e-19cc9d89892b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:00:49.114453 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c72bb13-d792-4f8a-880e-19cc9d89892b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:49.116173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c72bb13-d792-4f8a-880e-19cc9d89892b admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:49.148105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 959/1922] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:00:49.149554 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7813890-8bb8-4f79-be2f-5e02787b7821 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:49.149834 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7813890-8bb8-4f79-be2f-5e02787b7821 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:49.151761 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7813890-8bb8-4f79-be2f-5e02787b7821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:49.159929 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb55e9-e5b0-4746-87c4-1c2c2a3ffd1b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:49.165936 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b4f9d-8e94-4e3c-a2a2-f20b4d954369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:49.166596 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b4f9d-8e94-4e3c-a2a2-f20b4d954369 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:49.166884 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b4f9d-8e94-4e3c-a2a2-f20b4d954369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:49.167164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b4f9d-8e94-4e3c-a2a2-f20b4d954369 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:49.167927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7b4f9d-8e94-4e3c-a2a2-f20b4d954369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:49.169785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7b4f9d-8e94-4e3c-a2a2-f20b4d954369 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:49.193722 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:49.199077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 964/1923] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:49.203002 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b4f9d-8e94-4e3c-a2a2-f20b4d954369 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:49.217995 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-a7813890-8bb8-4f79-be2f-5e02787b7821 None None] MFA Rules not processed for user `e35673ddf25f4d279f82940f47a69026`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:49.221100 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a7813890-8bb8-4f79-be2f-5e02787b7821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:49.221398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b4f9d-8e94-4e3c-a2a2-f20b4d954369 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:49.231497 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334d3b3-6fbd-49aa-a18d-7f2fea59ad5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:49.233379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334d3b3-6fbd-49aa-a18d-7f2fea59ad5e admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:49.233913 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334d3b3-6fbd-49aa-a18d-7f2fea59ad5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:49.234357 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334d3b3-6fbd-49aa-a18d-7f2fea59ad5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:49.235448 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3334d3b3-6fbd-49aa-a18d-7f2fea59ad5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:49.240562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3334d3b3-6fbd-49aa-a18d-7f2fea59ad5e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:49.270528 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:49.280733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 965/1924] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:00:49 2022] POST /identity/v3/projects => generated 373 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:49.281832 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334d3b3-6fbd-49aa-a18d-7f2fea59ad5e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:49.401560 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 960/1925] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:49 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:49.402701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7813890-8bb8-4f79-be2f-5e02787b7821 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:49.418642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334d3b3-6fbd-49aa-a18d-7f2fea59ad5e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:49.428326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7813890-8bb8-4f79-be2f-5e02787b7821 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:49.433024 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f305f127-a83f-4271-8a5a-8d5e5330b406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:49.434930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305f127-a83f-4271-8a5a-8d5e5330b406 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:49.435869 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599d301-a75b-45c8-aee8-56bda596d49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:49.436281 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305f127-a83f-4271-8a5a-8d5e5330b406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:49.436527 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599d301-a75b-45c8-aee8-56bda596d49d admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:49.436813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599d301-a75b-45c8-aee8-56bda596d49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:49.437088 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599d301-a75b-45c8-aee8-56bda596d49d admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:49.437777 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a599d301-a75b-45c8-aee8-56bda596d49d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:49.439804 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305f127-a83f-4271-8a5a-8d5e5330b406 admin admin] PATH_INFO: `/v3/projects/ef2dac306deb47d19df60262012c7fb8/users/a1eae282478c434ebbe002513727d47b/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:49.444013 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a599d301-a75b-45c8-aee8-56bda596d49d admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:49.485028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f305f127-a83f-4271-8a5a-8d5e5330b406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef2dac306deb47d19df60262012c7fb8, user_id=a1eae282478c434ebbe002513727d47b, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:49.487248 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 961/1926] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:00:49 2022] POST /identity/v3/users => generated 287 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:49.488330 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599d301-a75b-45c8-aee8-56bda596d49d admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:49.489583 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f305f127-a83f-4271-8a5a-8d5e5330b406 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:49.501219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a599d301-a75b-45c8-aee8-56bda596d49d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:49.519951 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 966/1927] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:49 2022] PUT /identity/v3/projects/ef2dac306deb47d19df60262012c7fb8/users/a1eae282478c434ebbe002513727d47b/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:49.521869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f305f127-a83f-4271-8a5a-8d5e5330b406 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:49.668278 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599d301-a75b-45c8-aee8-56bda596d49d admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:49.709080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f305f127-a83f-4271-8a5a-8d5e5330b406 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:49.732177 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e927f59-65cd-4a94-9ea5-fc5e4f345e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:49.732996 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e927f59-65cd-4a94-9ea5-fc5e4f345e8b admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:49.733603 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e927f59-65cd-4a94-9ea5-fc5e4f345e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:49.733986 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e927f59-65cd-4a94-9ea5-fc5e4f345e8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:49.734920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e927f59-65cd-4a94-9ea5-fc5e4f345e8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:49.740306 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e927f59-65cd-4a94-9ea5-fc5e4f345e8b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:49.765082 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 967/1928] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:49.772423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e927f59-65cd-4a94-9ea5-fc5e4f345e8b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:49.785016 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39474d22-37f3-44b1-801f-a5d50a677a76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:49.786404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39474d22-37f3-44b1-801f-a5d50a677a76 service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:49.787169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39474d22-37f3-44b1-801f-a5d50a677a76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:49.787169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39474d22-37f3-44b1-801f-a5d50a677a76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:49.814556 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e927f59-65cd-4a94-9ea5-fc5e4f345e8b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:49.823432 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e115a-7de7-4e9e-b32c-5cc5d30de820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:49.824120 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e115a-7de7-4e9e-b32c-5cc5d30de820 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:49.824403 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e115a-7de7-4e9e-b32c-5cc5d30de820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:49.824678 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e115a-7de7-4e9e-b32c-5cc5d30de820 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:49.825620 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e115a-7de7-4e9e-b32c-5cc5d30de820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:49.827386 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e115a-7de7-4e9e-b32c-5cc5d30de820 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:49.852219 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 968/1929] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:49.853234 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e115a-7de7-4e9e-b32c-5cc5d30de820 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:49.855039 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:49.875713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e115a-7de7-4e9e-b32c-5cc5d30de820 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:49.876259 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39474d22-37f3-44b1-801f-a5d50a677a76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:49.878538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39474d22-37f3-44b1-801f-a5d50a677a76 service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:49.893312 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43344b4-9810-4a59-8c9e-dc00aa85c492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:49.893718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 962/1930] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:49 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:49.894325 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43344b4-9810-4a59-8c9e-dc00aa85c492 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:49.894613 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43344b4-9810-4a59-8c9e-dc00aa85c492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:49.894876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43344b4-9810-4a59-8c9e-dc00aa85c492 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:49.895051 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39474d22-37f3-44b1-801f-a5d50a677a76 service cinder] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:49.895609 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43344b4-9810-4a59-8c9e-dc00aa85c492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:49.897360 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43344b4-9810-4a59-8c9e-dc00aa85c492 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:49.911794 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39474d22-37f3-44b1-801f-a5d50a677a76 service cinder] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:49.919188 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef3332d-a3c4-4d0c-91c1-142cd943963b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:49.920206 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3332d-a3c4-4d0c-91c1-142cd943963b admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:49.920594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 969/1931] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:49.920703 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3332d-a3c4-4d0c-91c1-142cd943963b admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:49.921101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3332d-a3c4-4d0c-91c1-142cd943963b admin admin] PATH_INFO: `/v3/projects/ef2dac306deb47d19df60262012c7fb8/users/a1eae282478c434ebbe002513727d47b/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:49.921539 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43344b4-9810-4a59-8c9e-dc00aa85c492 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:49.923450 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef3332d-a3c4-4d0c-91c1-142cd943963b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef2dac306deb47d19df60262012c7fb8, user_id=a1eae282478c434ebbe002513727d47b, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:49.929238 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef3332d-a3c4-4d0c-91c1-142cd943963b admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:49.933040 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43344b4-9810-4a59-8c9e-dc00aa85c492 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:49.939226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36cbaff-13b7-4792-a8ee-b3ed7380cb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:49.940035 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36cbaff-13b7-4792-a8ee-b3ed7380cb78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:49.940319 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36cbaff-13b7-4792-a8ee-b3ed7380cb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:49.940598 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36cbaff-13b7-4792-a8ee-b3ed7380cb78 admin admin] PATH_INFO: `/v3/projects/7e58d3452a0d4be6a5624eff149d7b4d/users/b6e008acbc194b1484f26fc19d8c7a89/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:49.948542 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 963/1932] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:49 2022] PUT /identity/v3/projects/ef2dac306deb47d19df60262012c7fb8/users/a1eae282478c434ebbe002513727d47b/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:49.964943 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36cbaff-13b7-4792-a8ee-b3ed7380cb78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e58d3452a0d4be6a5624eff149d7b4d, user_id=b6e008acbc194b1484f26fc19d8c7a89, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:49.966620 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aab179-b0c7-4a7b-a10f-2d83b4a68a05 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:49.968027 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aab179-b0c7-4a7b-a10f-2d83b4a68a05 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:49.968725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36cbaff-13b7-4792-a8ee-b3ed7380cb78 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:49.968934 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aab179-b0c7-4a7b-a10f-2d83b4a68a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:50.008472 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 970/1933] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:49 2022] PUT /identity/v3/projects/7e58d3452a0d4be6a5624eff149d7b4d/users/b6e008acbc194b1484f26fc19d8c7a89/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:50.010761 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0bc4e-3680-40c6-97fa-787f47efe221 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:50.011151 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0bc4e-3680-40c6-97fa-787f47efe221 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:50.011634 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0bc4e-3680-40c6-97fa-787f47efe221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:50.039958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-58aab179-b0c7-4a7b-a10f-2d83b4a68a05 None None] MFA Rules not processed for user `a1eae282478c434ebbe002513727d47b`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:50.041488 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-58aab179-b0c7-4a7b-a10f-2d83b4a68a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:50.086010 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-dde0bc4e-3680-40c6-97fa-787f47efe221 None None] MFA Rules not processed for user `b6e008acbc194b1484f26fc19d8c7a89`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:50.088123 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dde0bc4e-3680-40c6-97fa-787f47efe221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:50.139393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:50.172401 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 964/1934] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:49 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:50.240442 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aab179-b0c7-4a7b-a10f-2d83b4a68a05 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:50.251777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 971/1935] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:50.277718 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0bc4e-3680-40c6-97fa-787f47efe221 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:50.344058 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aab179-b0c7-4a7b-a10f-2d83b4a68a05 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:50.350648 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60737a8e-b541-471e-9439-c4aa9fe04d84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:50.351296 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60737a8e-b541-471e-9439-c4aa9fe04d84 service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:50.351607 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60737a8e-b541-471e-9439-c4aa9fe04d84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:50.351877 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60737a8e-b541-471e-9439-c4aa9fe04d84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:50.377949 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0bc4e-3680-40c6-97fa-787f47efe221 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:50.384421 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b658f5-daed-48a8-8cc1-f4c1693b0dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:50.385170 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b658f5-daed-48a8-8cc1-f4c1693b0dd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:50.385499 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b658f5-daed-48a8-8cc1-f4c1693b0dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:50.385781 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b658f5-daed-48a8-8cc1-f4c1693b0dd2 admin admin] PATH_INFO: `/v3/users/8aeff3e2c3dd45d487af606dfd2fa12d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:50.387115 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b658f5-daed-48a8-8cc1-f4c1693b0dd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8aeff3e2c3dd45d487af606dfd2fa12d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:50.389647 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b658f5-daed-48a8-8cc1-f4c1693b0dd2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:50.478090 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60737a8e-b541-471e-9439-c4aa9fe04d84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:50.479117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-74b658f5-daed-48a8-8cc1-f4c1693b0dd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8aeff3e2c3dd45d487af606dfd2fa12d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:50.479968 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60737a8e-b541-471e-9439-c4aa9fe04d84 service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:50.492395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-74b658f5-daed-48a8-8cc1-f4c1693b0dd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8aeff3e2c3dd45d487af606dfd2fa12d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:50.493379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-74b658f5-daed-48a8-8cc1-f4c1693b0dd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8aeff3e2c3dd45d487af606dfd2fa12d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:50.508568 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-74b658f5-daed-48a8-8cc1-f4c1693b0dd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8aeff3e2c3dd45d487af606dfd2fa12d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:50.509368 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-74b658f5-daed-48a8-8cc1-f4c1693b0dd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8aeff3e2c3dd45d487af606dfd2fa12d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:50.513682 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 965/1936] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:50.515014 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60737a8e-b541-471e-9439-c4aa9fe04d84 service cinder] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:50.517442 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-60737a8e-b541-471e-9439-c4aa9fe04d84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:50.540733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60737a8e-b541-471e-9439-c4aa9fe04d84 service cinder] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:50.564217 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28435817-1f4f-411d-b16c-1548ed580905 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:50.565282 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28435817-1f4f-411d-b16c-1548ed580905 service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:50.565719 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28435817-1f4f-411d-b16c-1548ed580905 service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:50.566244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28435817-1f4f-411d-b16c-1548ed580905 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:50.568591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 972/1937] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:00:50 2022] DELETE /identity/v3/users/8aeff3e2c3dd45d487af606dfd2fa12d => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:00:50.571057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-28435817-1f4f-411d-b16c-1548ed580905 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:50.574821 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4644d9-bf92-4b04-88f9-00eef79bc7a9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:50.575238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4644d9-bf92-4b04-88f9-00eef79bc7a9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:50.575737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4644d9-bf92-4b04-88f9-00eef79bc7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:50.586794 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28435817-1f4f-411d-b16c-1548ed580905 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:50.589049 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28435817-1f4f-411d-b16c-1548ed580905 service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:50.596048 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:50.609230 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 966/1938] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:50.617571 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1f4644d9-bf92-4b04-88f9-00eef79bc7a9 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:50.619226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1f4644d9-bf92-4b04-88f9-00eef79bc7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:50.636876 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28435817-1f4f-411d-b16c-1548ed580905 service cinder] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:50.639121 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-28435817-1f4f-411d-b16c-1548ed580905 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:50.660506 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28435817-1f4f-411d-b16c-1548ed580905 service cinder] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:50.732026 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:50.777579 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 973/1939] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:50.789682 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4644d9-bf92-4b04-88f9-00eef79bc7a9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:50.792690 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1f4644d9-bf92-4b04-88f9-00eef79bc7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:50.840209 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473861d3-e396-44b1-9fbf-3c4543ff1ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:50.841210 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473861d3-e396-44b1-9fbf-3c4543ff1ab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:50.841612 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473861d3-e396-44b1-9fbf-3c4543ff1ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:50.841992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473861d3-e396-44b1-9fbf-3c4543ff1ab3 admin admin] PATH_INFO: `/v3/users/b6e008acbc194b1484f26fc19d8c7a89` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:50.847172 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473861d3-e396-44b1-9fbf-3c4543ff1ab3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6e008acbc194b1484f26fc19d8c7a89)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:50.850306 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473861d3-e396-44b1-9fbf-3c4543ff1ab3 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:50.998249 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-473861d3-e396-44b1-9fbf-3c4543ff1ab3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6e008acbc194b1484f26fc19d8c7a89'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:51.023233 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4644d9-bf92-4b04-88f9-00eef79bc7a9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:51.035777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b66f6-7b3a-43a6-8778-5b126628560e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:51.037510 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b66f6-7b3a-43a6-8778-5b126628560e service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:51.038005 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-473861d3-e396-44b1-9fbf-3c4543ff1ab3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6e008acbc194b1484f26fc19d8c7a89'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:51.039207 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b66f6-7b3a-43a6-8778-5b126628560e service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:51.041269 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b66f6-7b3a-43a6-8778-5b126628560e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:51.055663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-473861d3-e396-44b1-9fbf-3c4543ff1ab3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6e008acbc194b1484f26fc19d8c7a89'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:51.056965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-473861d3-e396-44b1-9fbf-3c4543ff1ab3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6e008acbc194b1484f26fc19d8c7a89'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:51.058002 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-473861d3-e396-44b1-9fbf-3c4543ff1ab3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6e008acbc194b1484f26fc19d8c7a89'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:51.070241 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b66f6-7b3a-43a6-8778-5b126628560e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:51.077262 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b66f6-7b3a-43a6-8778-5b126628560e service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:51.114752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 974/1940] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:51.117306 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 967/1941] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:00:50 2022] DELETE /identity/v3/users/b6e008acbc194b1484f26fc19d8c7a89 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:00:51.122644 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdeb7b1-57c5-40c5-bae3-517f8203db76 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:51.123247 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdeb7b1-57c5-40c5-bae3-517f8203db76 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:51.123902 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdeb7b1-57c5-40c5-bae3-517f8203db76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:51.160499 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1bdeb7b1-57c5-40c5-bae3-517f8203db76 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:51.160499 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1bdeb7b1-57c5-40c5-bae3-517f8203db76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:51.374237 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 975/1942] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:51.389900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473861d3-e396-44b1-9fbf-3c4543ff1ab3 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:51.389900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-473861d3-e396-44b1-9fbf-3c4543ff1ab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:51.527976 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473861d3-e396-44b1-9fbf-3c4543ff1ab3 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:51.547980 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec43f77-ab6f-4a7b-937c-ebd846f2a938 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:51.548938 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec43f77-ab6f-4a7b-937c-ebd846f2a938 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:51.549370 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec43f77-ab6f-4a7b-937c-ebd846f2a938 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:51.549840 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec43f77-ab6f-4a7b-937c-ebd846f2a938 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:51.580229 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec43f77-ab6f-4a7b-937c-ebd846f2a938 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:51.581825 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec43f77-ab6f-4a7b-937c-ebd846f2a938 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:51.602508 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 968/1943] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:52.603721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdeb7b1-57c5-40c5-bae3-517f8203db76 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:52.605434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1bdeb7b1-57c5-40c5-bae3-517f8203db76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:52.621487 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdeb7b1-57c5-40c5-bae3-517f8203db76 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:52.627304 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a6590-cdc9-47fd-9d16-a7e879ea6729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:52.628047 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a6590-cdc9-47fd-9d16-a7e879ea6729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:52.628356 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a6590-cdc9-47fd-9d16-a7e879ea6729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:52.628656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a6590-cdc9-47fd-9d16-a7e879ea6729 admin admin] PATH_INFO: `/v3/projects/7e58d3452a0d4be6a5624eff149d7b4d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:52.629881 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a6590-cdc9-47fd-9d16-a7e879ea6729 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e58d3452a0d4be6a5624eff149d7b4d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:52.632146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a6590-cdc9-47fd-9d16-a7e879ea6729 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:52.663139 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec43f77-ab6f-4a7b-937c-ebd846f2a938 service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:52.664628 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-6ec43f77-ab6f-4a7b-937c-ebd846f2a938 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:52.688032 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec43f77-ab6f-4a7b-937c-ebd846f2a938 service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:52.708150 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b99a6590-cdc9-47fd-9d16-a7e879ea6729 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e58d3452a0d4be6a5624eff149d7b4d', 'request_id': 'req-b99a6590-cdc9-47fd-9d16-a7e879ea6729'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:52.718324 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b99a6590-cdc9-47fd-9d16-a7e879ea6729 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e58d3452a0d4be6a5624eff149d7b4d', 'request_id': 'req-b99a6590-cdc9-47fd-9d16-a7e879ea6729'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:52.733025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b99a6590-cdc9-47fd-9d16-a7e879ea6729 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e58d3452a0d4be6a5624eff149d7b4d', 'request_id': 'req-b99a6590-cdc9-47fd-9d16-a7e879ea6729'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:52.736896 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:52.749676 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b99a6590-cdc9-47fd-9d16-a7e879ea6729 admin admin] The token cache is being invalidate because project 7e58d3452a0d4be6a5624eff149d7b4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:00:52.750245 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b99a6590-cdc9-47fd-9d16-a7e879ea6729 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:52.750997 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b99a6590-cdc9-47fd-9d16-a7e879ea6729 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:52.756981 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 976/1944] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:00:52 2022] DELETE /identity/v3/projects/7e58d3452a0d4be6a5624eff149d7b4d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:52.759987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81139b00-4175-482a-b40f-3ad84757c9a8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:52.761038 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81139b00-4175-482a-b40f-3ad84757c9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:52.761471 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81139b00-4175-482a-b40f-3ad84757c9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:52.791338 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-81139b00-4175-482a-b40f-3ad84757c9a8 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:52.792383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341a64b-22a0-42b6-a1a7-3ae2114c2ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:52.792992 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-81139b00-4175-482a-b40f-3ad84757c9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:52.793175 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341a64b-22a0-42b6-a1a7-3ae2114c2ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:52.793521 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341a64b-22a0-42b6-a1a7-3ae2114c2ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:52.793872 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341a64b-22a0-42b6-a1a7-3ae2114c2ff6 admin admin] PATH_INFO: `/v3/projects/da7821e3dcbb4ddb9e9ed57e171dee6f` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:52.795104 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6341a64b-22a0-42b6-a1a7-3ae2114c2ff6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da7821e3dcbb4ddb9e9ed57e171dee6f)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:52.797232 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6341a64b-22a0-42b6-a1a7-3ae2114c2ff6 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:52.804894 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 977/1945] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:52.811845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd178a-5e66-4545-9878-1cf082787625 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:52.812240 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd178a-5e66-4545-9878-1cf082787625 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:52.812603 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd178a-5e66-4545-9878-1cf082787625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:52.821358 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:52.834533 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-35cd178a-5e66-4545-9878-1cf082787625 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:52.836674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-35cd178a-5e66-4545-9878-1cf082787625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:52.854405 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6341a64b-22a0-42b6-a1a7-3ae2114c2ff6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da7821e3dcbb4ddb9e9ed57e171dee6f', 'request_id': 'req-6341a64b-22a0-42b6-a1a7-3ae2114c2ff6'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:52.868612 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6341a64b-22a0-42b6-a1a7-3ae2114c2ff6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da7821e3dcbb4ddb9e9ed57e171dee6f', 'request_id': 'req-6341a64b-22a0-42b6-a1a7-3ae2114c2ff6'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:52.877224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6341a64b-22a0-42b6-a1a7-3ae2114c2ff6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da7821e3dcbb4ddb9e9ed57e171dee6f', 'request_id': 'req-6341a64b-22a0-42b6-a1a7-3ae2114c2ff6'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:52.890481 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6341a64b-22a0-42b6-a1a7-3ae2114c2ff6 admin admin] The token cache is being invalidate because project da7821e3dcbb4ddb9e9ed57e171dee6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:00:52.890783 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6341a64b-22a0-42b6-a1a7-3ae2114c2ff6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:52.891423 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6341a64b-22a0-42b6-a1a7-3ae2114c2ff6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:52.896288 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 969/1946] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:00:52 2022] DELETE /identity/v3/projects/da7821e3dcbb4ddb9e9ed57e171dee6f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:00:52.897284 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341a64b-22a0-42b6-a1a7-3ae2114c2ff6 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:52.898725 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-6341a64b-22a0-42b6-a1a7-3ae2114c2ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:52.913418 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 978/1947] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:00:52.919905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd178a-5e66-4545-9878-1cf082787625 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:52.930841 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd178a-5e66-4545-9878-1cf082787625 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:52.942303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7511363-d65e-4048-9e91-0c9720b505fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:52.943134 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7511363-d65e-4048-9e91-0c9720b505fc admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:52.943521 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7511363-d65e-4048-9e91-0c9720b505fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:52.943918 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7511363-d65e-4048-9e91-0c9720b505fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:52.945348 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7511363-d65e-4048-9e91-0c9720b505fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:00:52.945753 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7511363-d65e-4048-9e91-0c9720b505fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:52.948042 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7511363-d65e-4048-9e91-0c9720b505fc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:52.964691 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 979/1948] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:00:52.972832 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7511363-d65e-4048-9e91-0c9720b505fc admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:52.984983 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7511363-d65e-4048-9e91-0c9720b505fc admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:52.991512 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eabe62c-1483-41dd-a606-cd5734e0e3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:52.991512 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eabe62c-1483-41dd-a606-cd5734e0e3d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:52.991788 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eabe62c-1483-41dd-a606-cd5734e0e3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:52.991940 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eabe62c-1483-41dd-a606-cd5734e0e3d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:52.992612 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eabe62c-1483-41dd-a606-cd5734e0e3d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:52.994560 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eabe62c-1483-41dd-a606-cd5734e0e3d2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:53.025972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341a64b-22a0-42b6-a1a7-3ae2114c2ff6 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:53.037174 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4854ada-90e6-41a1-a5b3-19469f7b8a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:53.037854 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4854ada-90e6-41a1-a5b3-19469f7b8a39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:53.038143 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4854ada-90e6-41a1-a5b3-19469f7b8a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:53.038424 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4854ada-90e6-41a1-a5b3-19469f7b8a39 admin admin] PATH_INFO: `/v3/users/e35673ddf25f4d279f82940f47a69026` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:53.039448 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 980/1949] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:00:52 2022] POST /identity/v3/projects => generated 375 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:53.042741 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4854ada-90e6-41a1-a5b3-19469f7b8a39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e35673ddf25f4d279f82940f47a69026)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:53.043234 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eabe62c-1483-41dd-a606-cd5734e0e3d2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:53.044776 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4854ada-90e6-41a1-a5b3-19469f7b8a39 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:53.109611 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eabe62c-1483-41dd-a606-cd5734e0e3d2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:53.118871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a98ec-bdc8-4ba6-bb78-b5c2a4de5cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:53.119540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a98ec-bdc8-4ba6-bb78-b5c2a4de5cab admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:53.119862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a98ec-bdc8-4ba6-bb78-b5c2a4de5cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:53.120169 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a98ec-bdc8-4ba6-bb78-b5c2a4de5cab admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:53.120872 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a98ec-bdc8-4ba6-bb78-b5c2a4de5cab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:53.122865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a98ec-bdc8-4ba6-bb78-b5c2a4de5cab admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:53.126175 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-d4854ada-90e6-41a1-a5b3-19469f7b8a39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e35673ddf25f4d279f82940f47a69026'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:53.130906 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:53.134236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-d4854ada-90e6-41a1-a5b3-19469f7b8a39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e35673ddf25f4d279f82940f47a69026'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:53.143641 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-d4854ada-90e6-41a1-a5b3-19469f7b8a39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e35673ddf25f4d279f82940f47a69026'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:53.144291 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-d4854ada-90e6-41a1-a5b3-19469f7b8a39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e35673ddf25f4d279f82940f47a69026'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:53.145061 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-d4854ada-90e6-41a1-a5b3-19469f7b8a39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e35673ddf25f4d279f82940f47a69026'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:53.147581 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 981/1950] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:00:53 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:00:53.151543 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a98ec-bdc8-4ba6-bb78-b5c2a4de5cab admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:53.153298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-422a98ec-bdc8-4ba6-bb78-b5c2a4de5cab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:53.163591 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 970/1951] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:00:52 2022] DELETE /identity/v3/users/e35673ddf25f4d279f82940f47a69026 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:00:53.165412 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a98ec-bdc8-4ba6-bb78-b5c2a4de5cab admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:53.228969 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:53.241099 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f773c2-e2e2-411c-bc72-d80ab7460511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:53.243771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f773c2-e2e2-411c-bc72-d80ab7460511 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:53.243771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f773c2-e2e2-411c-bc72-d80ab7460511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:53.243771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f773c2-e2e2-411c-bc72-d80ab7460511 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:53.244588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f773c2-e2e2-411c-bc72-d80ab7460511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:53.246413 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f773c2-e2e2-411c-bc72-d80ab7460511 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:53.262117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 982/1952] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:53.266001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4854ada-90e6-41a1-a5b3-19469f7b8a39 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:53.276487 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4854ada-90e6-41a1-a5b3-19469f7b8a39 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:53.281533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7871e-f5ac-4c06-ba0a-03fe8215bf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:53.282168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7871e-f5ac-4c06-ba0a-03fe8215bf8f admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:53.282451 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7871e-f5ac-4c06-ba0a-03fe8215bf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:53.282722 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7871e-f5ac-4c06-ba0a-03fe8215bf8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:53.283447 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae7871e-f5ac-4c06-ba0a-03fe8215bf8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:53.285399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae7871e-f5ac-4c06-ba0a-03fe8215bf8f admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:53.315535 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 971/1953] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:53.320855 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f773c2-e2e2-411c-bc72-d80ab7460511 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:53.339307 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f773c2-e2e2-411c-bc72-d80ab7460511 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:53.350838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459a450-9be5-4c90-9eb9-7bbf10404176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:53.351624 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459a450-9be5-4c90-9eb9-7bbf10404176 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:53.351914 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459a450-9be5-4c90-9eb9-7bbf10404176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:53.352240 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459a450-9be5-4c90-9eb9-7bbf10404176 admin admin] PATH_INFO: `/v3/projects/54ad2fdbdee748cba641763fd7ccd878/users/61d57ac613b34adea593c68b13578d8d/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:53.373024 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a459a450-9be5-4c90-9eb9-7bbf10404176 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54ad2fdbdee748cba641763fd7ccd878, user_id=61d57ac613b34adea593c68b13578d8d, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:53.376171 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a459a450-9be5-4c90-9eb9-7bbf10404176 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:53.399195 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 983/1954] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:53 2022] PUT /identity/v3/projects/54ad2fdbdee748cba641763fd7ccd878/users/61d57ac613b34adea593c68b13578d8d/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:53.409898 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eaf2c4-00e2-4fef-a52a-748e998dc6e8 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:53.410847 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eaf2c4-00e2-4fef-a52a-748e998dc6e8 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:53.411341 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eaf2c4-00e2-4fef-a52a-748e998dc6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:53.441189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-12eaf2c4-00e2-4fef-a52a-748e998dc6e8 None None] MFA Rules not processed for user `61d57ac613b34adea593c68b13578d8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:53.442665 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-12eaf2c4-00e2-4fef-a52a-748e998dc6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:53.510181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 972/1955] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:00:53.524801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459a450-9be5-4c90-9eb9-7bbf10404176 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:53.524801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a459a450-9be5-4c90-9eb9-7bbf10404176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:53.556173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eaf2c4-00e2-4fef-a52a-748e998dc6e8 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:53.557691 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-12eaf2c4-00e2-4fef-a52a-748e998dc6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:53.612885 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459a450-9be5-4c90-9eb9-7bbf10404176 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:53.616781 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b658d-3261-42ab-8045-0d1fa5e2eb6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:53.617452 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b658d-3261-42ab-8045-0d1fa5e2eb6a service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:53.618349 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b658d-3261-42ab-8045-0d1fa5e2eb6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:53.618623 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b658d-3261-42ab-8045-0d1fa5e2eb6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:53.629514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b658d-3261-42ab-8045-0d1fa5e2eb6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:53.631091 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b658d-3261-42ab-8045-0d1fa5e2eb6a service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:53.633759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eaf2c4-00e2-4fef-a52a-748e998dc6e8 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:53.638504 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6dfa1-65ea-4f15-936e-db3206de745c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:53.639178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6dfa1-65ea-4f15-936e-db3206de745c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:53.639542 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6dfa1-65ea-4f15-936e-db3206de745c admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:53.639950 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6dfa1-65ea-4f15-936e-db3206de745c admin admin] PATH_INFO: `/v3/projects/5d87219713724001927cd14519889d8a` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:53.641139 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6dfa1-65ea-4f15-936e-db3206de745c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d87219713724001927cd14519889d8a)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:53.641850 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 984/1956] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:53.643078 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6dfa1-65ea-4f15-936e-db3206de745c admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:53.705132 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6af6dfa1-65ea-4f15-936e-db3206de745c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d87219713724001927cd14519889d8a', 'request_id': 'req-6af6dfa1-65ea-4f15-936e-db3206de745c'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:53.709539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:53.718537 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6af6dfa1-65ea-4f15-936e-db3206de745c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d87219713724001927cd14519889d8a', 'request_id': 'req-6af6dfa1-65ea-4f15-936e-db3206de745c'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:53.732171 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6af6dfa1-65ea-4f15-936e-db3206de745c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d87219713724001927cd14519889d8a', 'request_id': 'req-6af6dfa1-65ea-4f15-936e-db3206de745c'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:53.750967 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6af6dfa1-65ea-4f15-936e-db3206de745c admin admin] The token cache is being invalidate because project 5d87219713724001927cd14519889d8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:00:53.751398 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6af6dfa1-65ea-4f15-936e-db3206de745c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:53.752514 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6af6dfa1-65ea-4f15-936e-db3206de745c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:00:53.756010 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 973/1957] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:00:53 2022] DELETE /identity/v3/projects/5d87219713724001927cd14519889d8a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:00:53.764958 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b102317-ec64-4642-b222-ce7a73fdd08d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:53.765283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b102317-ec64-4642-b222-ce7a73fdd08d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:53.765573 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b102317-ec64-4642-b222-ce7a73fdd08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:53.788007 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2b102317-ec64-4642-b222-ce7a73fdd08d None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:53.789226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2b102317-ec64-4642-b222-ce7a73fdd08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:53.874206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 985/1958] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:53.885317 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773b99d-42fa-46b1-b32b-485c164b3634 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:53.885618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773b99d-42fa-46b1-b32b-485c164b3634 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:53.885900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773b99d-42fa-46b1-b32b-485c164b3634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:53.910873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-4773b99d-42fa-46b1-b32b-485c164b3634 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:53.912136 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-4773b99d-42fa-46b1-b32b-485c164b3634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:53.921333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 974/1959] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:53.926584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b102317-ec64-4642-b222-ce7a73fdd08d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:53.938234 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b102317-ec64-4642-b222-ce7a73fdd08d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:53.944316 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524689a-1b11-432b-9a0d-bb17296152d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:53.945077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524689a-1b11-432b-9a0d-bb17296152d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:53.945467 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524689a-1b11-432b-9a0d-bb17296152d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:53.945847 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524689a-1b11-432b-9a0d-bb17296152d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:53.946878 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524689a-1b11-432b-9a0d-bb17296152d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:00:53.947288 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524689a-1b11-432b-9a0d-bb17296152d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:53.949166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524689a-1b11-432b-9a0d-bb17296152d4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:53.966047 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 986/1960] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:00:53.970629 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4773b99d-42fa-46b1-b32b-485c164b3634 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:53.982615 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4773b99d-42fa-46b1-b32b-485c164b3634 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:53.987920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe46b55-9c2a-4ce9-9248-4e63f3000906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:53.988583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe46b55-9c2a-4ce9-9248-4e63f3000906 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:53.988868 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe46b55-9c2a-4ce9-9248-4e63f3000906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:53.989145 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe46b55-9c2a-4ce9-9248-4e63f3000906 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:53.989860 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe46b55-9c2a-4ce9-9248-4e63f3000906 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:53.991936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe46b55-9c2a-4ce9-9248-4e63f3000906 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:54.031113 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 975/1961] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:00:53 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:00:54.032563 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524689a-1b11-432b-9a0d-bb17296152d4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:54.044605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524689a-1b11-432b-9a0d-bb17296152d4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:54.114895 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08937896-771f-4d25-b7e5-c9552a5b8de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:54.116807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08937896-771f-4d25-b7e5-c9552a5b8de0 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:54.116807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08937896-771f-4d25-b7e5-c9552a5b8de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:54.116807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08937896-771f-4d25-b7e5-c9552a5b8de0 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:54.117297 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08937896-771f-4d25-b7e5-c9552a5b8de0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:54.120191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08937896-771f-4d25-b7e5-c9552a5b8de0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:54.129919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:54.152442 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 987/1962] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:00:54 2022] POST /identity/v3/users => generated 286 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:54.156782 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe46b55-9c2a-4ce9-9248-4e63f3000906 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:54.180112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe46b55-9c2a-4ce9-9248-4e63f3000906 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:54.185198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10f4be-edaf-4b77-bef6-33c431504b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:54.185916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10f4be-edaf-4b77-bef6-33c431504b3b admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:54.186208 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10f4be-edaf-4b77-bef6-33c431504b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:54.186481 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10f4be-edaf-4b77-bef6-33c431504b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:54.187189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10f4be-edaf-4b77-bef6-33c431504b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:54.188827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10f4be-edaf-4b77-bef6-33c431504b3b admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:54.200016 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 976/1963] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:54.204449 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08937896-771f-4d25-b7e5-c9552a5b8de0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:54.214888 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08937896-771f-4d25-b7e5-c9552a5b8de0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:54.220129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2cdb2d-9a48-45e2-80c8-39f3a2def6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:54.221052 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cdb2d-9a48-45e2-80c8-39f3a2def6c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:54.221052 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cdb2d-9a48-45e2-80c8-39f3a2def6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:54.221333 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cdb2d-9a48-45e2-80c8-39f3a2def6c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:54.222081 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2cdb2d-9a48-45e2-80c8-39f3a2def6c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:54.223999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2cdb2d-9a48-45e2-80c8-39f3a2def6c3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:54.235858 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 988/1964] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:00:54.240284 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10f4be-edaf-4b77-bef6-33c431504b3b admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:54.250743 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10f4be-edaf-4b77-bef6-33c431504b3b admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:54.255893 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcddc055-77a1-4510-b455-e334d36095b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:54.256582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcddc055-77a1-4510-b455-e334d36095b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:54.256865 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcddc055-77a1-4510-b455-e334d36095b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:54.257146 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcddc055-77a1-4510-b455-e334d36095b2 admin admin] PATH_INFO: `/v3/projects/a92fc94054a44582b774fcf682f96fcf/users/af96dc0075324b0690cdd6b69b1b9e4d/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:54.275452 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcddc055-77a1-4510-b455-e334d36095b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a92fc94054a44582b774fcf682f96fcf, user_id=af96dc0075324b0690cdd6b69b1b9e4d, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:54.278202 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcddc055-77a1-4510-b455-e334d36095b2 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:54.293528 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 977/1965] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:54 2022] PUT /identity/v3/projects/a92fc94054a44582b774fcf682f96fcf/users/af96dc0075324b0690cdd6b69b1b9e4d/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:54.303627 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dee135-53a3-4a30-80dc-d7e3b76dc0d9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:54.303946 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dee135-53a3-4a30-80dc-d7e3b76dc0d9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:54.304207 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dee135-53a3-4a30-80dc-d7e3b76dc0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:54.332972 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b6dee135-53a3-4a30-80dc-d7e3b76dc0d9 None None] MFA Rules not processed for user `af96dc0075324b0690cdd6b69b1b9e4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:54.334036 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b6dee135-53a3-4a30-80dc-d7e3b76dc0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:54.410690 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 989/1966] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:54.415667 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcddc055-77a1-4510-b455-e334d36095b2 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:54.481494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcddc055-77a1-4510-b455-e334d36095b2 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:54.486389 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f3682-68f2-459d-8a73-7d725282dbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:54.487042 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f3682-68f2-459d-8a73-7d725282dbc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:54.488187 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f3682-68f2-459d-8a73-7d725282dbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:54.488187 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f3682-68f2-459d-8a73-7d725282dbc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:54.488441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f3682-68f2-459d-8a73-7d725282dbc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:54.490347 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f3682-68f2-459d-8a73-7d725282dbc1 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:54.520585 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 978/1967] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:00:54 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:54.524481 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dee135-53a3-4a30-80dc-d7e3b76dc0d9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:54.535572 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dee135-53a3-4a30-80dc-d7e3b76dc0d9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:54.597886 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d68132-e4c1-4054-a17f-fbd971d3f679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:54.598446 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d68132-e4c1-4054-a17f-fbd971d3f679 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:54.598734 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d68132-e4c1-4054-a17f-fbd971d3f679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:54.599033 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d68132-e4c1-4054-a17f-fbd971d3f679 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:54.599765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d68132-e4c1-4054-a17f-fbd971d3f679 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:54.601733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d68132-e4c1-4054-a17f-fbd971d3f679 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:54.625316 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 990/1968] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:00:54 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:00:54.629776 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f3682-68f2-459d-8a73-7d725282dbc1 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:54.640658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f3682-68f2-459d-8a73-7d725282dbc1 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:54.647210 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d320c-bf56-4eb9-b436-10cd9cac5429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:54.647989 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d320c-bf56-4eb9-b436-10cd9cac5429 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:54.648265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d320c-bf56-4eb9-b436-10cd9cac5429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:54.648534 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d320c-bf56-4eb9-b436-10cd9cac5429 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:54.649240 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d320c-bf56-4eb9-b436-10cd9cac5429 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:54.650871 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d320c-bf56-4eb9-b436-10cd9cac5429 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:54.662410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 979/1969] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:54.666966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d68132-e4c1-4054-a17f-fbd971d3f679 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:54.677964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d68132-e4c1-4054-a17f-fbd971d3f679 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:54.683194 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc258f-fefa-4511-be10-cc18dd3ca6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:54.683931 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc258f-fefa-4511-be10-cc18dd3ca6a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:54.684235 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc258f-fefa-4511-be10-cc18dd3ca6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:54.684515 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc258f-fefa-4511-be10-cc18dd3ca6a7 admin admin] PATH_INFO: `/v3/projects/e02d6af390084c4f98dabd459da5a1da/users/94d061c1a4e641e38a842f3eed85d77b/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:54.702607 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc258f-fefa-4511-be10-cc18dd3ca6a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e02d6af390084c4f98dabd459da5a1da, user_id=94d061c1a4e641e38a842f3eed85d77b, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:54.705444 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc258f-fefa-4511-be10-cc18dd3ca6a7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:54.723450 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 991/1970] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:54 2022] PUT /identity/v3/projects/e02d6af390084c4f98dabd459da5a1da/users/94d061c1a4e641e38a842f3eed85d77b/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:54.724735 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d320c-bf56-4eb9-b436-10cd9cac5429 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:54.735204 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d320c-bf56-4eb9-b436-10cd9cac5429 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:54.779264 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:54.794818 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cff8d9-af5c-4ea3-9829-c6cdcaff59ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:54.795472 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cff8d9-af5c-4ea3-9829-c6cdcaff59ca admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:54.795764 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cff8d9-af5c-4ea3-9829-c6cdcaff59ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:54.796078 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cff8d9-af5c-4ea3-9829-c6cdcaff59ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:54.796763 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cff8d9-af5c-4ea3-9829-c6cdcaff59ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:54.798416 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cff8d9-af5c-4ea3-9829-c6cdcaff59ca admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:54.809429 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 980/1971] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:00:54.813702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc258f-fefa-4511-be10-cc18dd3ca6a7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:54.824655 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc258f-fefa-4511-be10-cc18dd3ca6a7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:54.830198 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f87f9-bca4-4736-ada2-4b76f1cf09c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:54.830885 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f87f9-bca4-4736-ada2-4b76f1cf09c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:54.831171 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f87f9-bca4-4736-ada2-4b76f1cf09c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:54.831482 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f87f9-bca4-4736-ada2-4b76f1cf09c9 admin admin] PATH_INFO: `/v3/projects/e02d6af390084c4f98dabd459da5a1da/users/94d061c1a4e641e38a842f3eed85d77b/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:54.834004 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050f87f9-bca4-4736-ada2-4b76f1cf09c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e02d6af390084c4f98dabd459da5a1da, user_id=94d061c1a4e641e38a842f3eed85d77b, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:54.836720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050f87f9-bca4-4736-ada2-4b76f1cf09c9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:54.847753 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 992/1972] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:00:54 2022] PUT /identity/v3/projects/e02d6af390084c4f98dabd459da5a1da/users/94d061c1a4e641e38a842f3eed85d77b/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:00:54.858211 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0fd5ed-c306-4c3b-b4da-548bc0e1e8ba None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:54.858591 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0fd5ed-c306-4c3b-b4da-548bc0e1e8ba None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:54.859008 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0fd5ed-c306-4c3b-b4da-548bc0e1e8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:54.893224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-8a0fd5ed-c306-4c3b-b4da-548bc0e1e8ba None None] MFA Rules not processed for user `94d061c1a4e641e38a842f3eed85d77b`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:54.894484 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-8a0fd5ed-c306-4c3b-b4da-548bc0e1e8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:54.964741 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 981/1973] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:00:54 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:54.976669 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f87f9-bca4-4736-ada2-4b76f1cf09c9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:54.978930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-050f87f9-bca4-4736-ada2-4b76f1cf09c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:55.009207 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:55.054213 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f87f9-bca4-4736-ada2-4b76f1cf09c9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:55.057966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac79f4-1c94-4352-a103-c2ebc4e725f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:55.058604 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac79f4-1c94-4352-a103-c2ebc4e725f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:55.058881 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac79f4-1c94-4352-a103-c2ebc4e725f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:55.059152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac79f4-1c94-4352-a103-c2ebc4e725f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:55.130309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac79f4-1c94-4352-a103-c2ebc4e725f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:55.131855 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac79f4-1c94-4352-a103-c2ebc4e725f3 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:55.143525 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 993/1974] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:56.247939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc367ee-69ab-4b88-bd28-4fd74d57af30 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:56.248716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc367ee-69ab-4b88-bd28-4fd74d57af30 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:56.248716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc367ee-69ab-4b88-bd28-4fd74d57af30 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:56.251135 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 982/1975] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jul 26 05:00:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 05:00:56.264738 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c00e2-e4c1-4470-8a60-700d787c6cce None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:56.266158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c00e2-e4c1-4470-8a60-700d787c6cce None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:56.266797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c00e2-e4c1-4470-8a60-700d787c6cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:56.295166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f41c00e2-e4c1-4470-8a60-700d787c6cce None None] MFA Rules not processed for user `2faeed9002ee4154bc980c146cd06bcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:00:56.295546 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f41c00e2-e4c1-4470-8a60-700d787c6cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:56.400963 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 994/1976] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jul 26 05:00:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:00:56.438463 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc367ee-69ab-4b88-bd28-4fd74d57af30 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:56.441325 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-bdc367ee-69ab-4b88-bd28-4fd74d57af30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:56.509367 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc367ee-69ab-4b88-bd28-4fd74d57af30 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:56.518458 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62af0e3-5600-44da-915c-027eecba394a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:56.519299 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62af0e3-5600-44da-915c-027eecba394a service nova] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:56.519677 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62af0e3-5600-44da-915c-027eecba394a service nova] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:56.519962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62af0e3-5600-44da-915c-027eecba394a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:56.534935 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62af0e3-5600-44da-915c-027eecba394a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:56.536337 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62af0e3-5600-44da-915c-027eecba394a service nova] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:56.559540 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 983/1977] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:00:56.945455 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c00e2-e4c1-4470-8a60-700d787c6cce None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:00:56.947293 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f41c00e2-e4c1-4470-8a60-700d787c6cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:56.968904 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c00e2-e4c1-4470-8a60-700d787c6cce None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:00:56.978001 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b3d5f-9941-44b3-a3ec-0ac3bdd2db44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:00:56.978666 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b3d5f-9941-44b3-a3ec-0ac3bdd2db44 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:00:56.978951 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b3d5f-9941-44b3-a3ec-0ac3bdd2db44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:00:56.979231 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b3d5f-9941-44b3-a3ec-0ac3bdd2db44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:00:56.985086 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c60b3d5f-9941-44b3-a3ec-0ac3bdd2db44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:00:57.102021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b3d5f-9941-44b3-a3ec-0ac3bdd2db44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:00:57.103487 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b3d5f-9941-44b3-a3ec-0ac3bdd2db44 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:00:57.109701 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:00:57.117872 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 995/1978] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:00:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:01:06.522620 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62af0e3-5600-44da-915c-027eecba394a service nova] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:06.610621 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62af0e3-5600-44da-915c-027eecba394a service nova] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:06.615462 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d7a0a-ff38-4309-84bc-c59230c694eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:06.616162 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d7a0a-ff38-4309-84bc-c59230c694eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:06.616448 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d7a0a-ff38-4309-84bc-c59230c694eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:06.616724 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d7a0a-ff38-4309-84bc-c59230c694eb admin admin] PATH_INFO: `/v3/users/af96dc0075324b0690cdd6b69b1b9e4d` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:06.617965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d7a0a-ff38-4309-84bc-c59230c694eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=af96dc0075324b0690cdd6b69b1b9e4d)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:06.619979 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d7a0a-ff38-4309-84bc-c59230c694eb admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:06.675035 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-7c7d7a0a-ff38-4309-84bc-c59230c694eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af96dc0075324b0690cdd6b69b1b9e4d'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:06.684142 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-7c7d7a0a-ff38-4309-84bc-c59230c694eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af96dc0075324b0690cdd6b69b1b9e4d'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:06.687026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:06.694132 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-7c7d7a0a-ff38-4309-84bc-c59230c694eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af96dc0075324b0690cdd6b69b1b9e4d'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:06.695129 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-7c7d7a0a-ff38-4309-84bc-c59230c694eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af96dc0075324b0690cdd6b69b1b9e4d'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:06.696306 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-7c7d7a0a-ff38-4309-84bc-c59230c694eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af96dc0075324b0690cdd6b69b1b9e4d'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:06.716770 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 984/1979] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:01:06 2022] DELETE /identity/v3/users/af96dc0075324b0690cdd6b69b1b9e4d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:01:06.722480 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6808e8b-69da-438a-b70e-15de1d2937c2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:06.723029 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6808e8b-69da-438a-b70e-15de1d2937c2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:06.723700 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6808e8b-69da-438a-b70e-15de1d2937c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:06.743999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c6808e8b-69da-438a-b70e-15de1d2937c2 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:06.746116 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c6808e8b-69da-438a-b70e-15de1d2937c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:06.833707 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 996/1980] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:01:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:01:06.845452 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d7a0a-ff38-4309-84bc-c59230c694eb admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:06.847432 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-7c7d7a0a-ff38-4309-84bc-c59230c694eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:06.961072 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d7a0a-ff38-4309-84bc-c59230c694eb admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:06.965286 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71579a46-2233-40e3-9b88-e659af931fba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:06.965845 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71579a46-2233-40e3-9b88-e659af931fba service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:06.966128 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71579a46-2233-40e3-9b88-e659af931fba service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:06.966406 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71579a46-2233-40e3-9b88-e659af931fba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:06.977658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71579a46-2233-40e3-9b88-e659af931fba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:06.979339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71579a46-2233-40e3-9b88-e659af931fba service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:06.991051 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 985/1981] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:01:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:01:07.806162 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6808e8b-69da-438a-b70e-15de1d2937c2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:07.807712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c6808e8b-69da-438a-b70e-15de1d2937c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:07.836001 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6808e8b-69da-438a-b70e-15de1d2937c2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:07.845228 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f394771-0f24-49d8-8c98-680a13c298b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:07.846348 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f394771-0f24-49d8-8c98-680a13c298b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:07.846730 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f394771-0f24-49d8-8c98-680a13c298b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:07.847240 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f394771-0f24-49d8-8c98-680a13c298b1 admin admin] PATH_INFO: `/v3/projects/a92fc94054a44582b774fcf682f96fcf` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:07.855838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f394771-0f24-49d8-8c98-680a13c298b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a92fc94054a44582b774fcf682f96fcf)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:07.858335 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f394771-0f24-49d8-8c98-680a13c298b1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:07.924826 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0f394771-0f24-49d8-8c98-680a13c298b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a92fc94054a44582b774fcf682f96fcf', 'request_id': 'req-0f394771-0f24-49d8-8c98-680a13c298b1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:07.932412 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0f394771-0f24-49d8-8c98-680a13c298b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a92fc94054a44582b774fcf682f96fcf', 'request_id': 'req-0f394771-0f24-49d8-8c98-680a13c298b1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:07.946976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0f394771-0f24-49d8-8c98-680a13c298b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a92fc94054a44582b774fcf682f96fcf', 'request_id': 'req-0f394771-0f24-49d8-8c98-680a13c298b1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:07.969210 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0f394771-0f24-49d8-8c98-680a13c298b1 admin admin] The token cache is being invalidate because project a92fc94054a44582b774fcf682f96fcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:01:07.969551 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0f394771-0f24-49d8-8c98-680a13c298b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:07.970263 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0f394771-0f24-49d8-8c98-680a13c298b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:07.975290 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 997/1982] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:01:07 2022] DELETE /identity/v3/projects/a92fc94054a44582b774fcf682f96fcf => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:01:07.976623 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71579a46-2233-40e3-9b88-e659af931fba service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:07.990090 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71579a46-2233-40e3-9b88-e659af931fba service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:08.064445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615d311-322a-42a3-bbc0-90791a4a795e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:08.065157 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615d311-322a-42a3-bbc0-90791a4a795e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:08.065435 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615d311-322a-42a3-bbc0-90791a4a795e admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:08.065708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615d311-322a-42a3-bbc0-90791a4a795e admin admin] PATH_INFO: `/v3/users/94d061c1a4e641e38a842f3eed85d77b` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:08.066925 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6615d311-322a-42a3-bbc0-90791a4a795e admin admin] RBAC: Authorizing `identity:delete_user(user_id=94d061c1a4e641e38a842f3eed85d77b)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:08.069024 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6615d311-322a-42a3-bbc0-90791a4a795e admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:08.121381 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6615d311-322a-42a3-bbc0-90791a4a795e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94d061c1a4e641e38a842f3eed85d77b'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:08.130839 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6615d311-322a-42a3-bbc0-90791a4a795e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94d061c1a4e641e38a842f3eed85d77b'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:08.142148 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6615d311-322a-42a3-bbc0-90791a4a795e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94d061c1a4e641e38a842f3eed85d77b'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:08.142779 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6615d311-322a-42a3-bbc0-90791a4a795e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94d061c1a4e641e38a842f3eed85d77b'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:08.143411 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6615d311-322a-42a3-bbc0-90791a4a795e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94d061c1a4e641e38a842f3eed85d77b'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:08.160430 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:08.169928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 986/1983] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:01:07 2022] DELETE /identity/v3/users/94d061c1a4e641e38a842f3eed85d77b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:01:08.572648 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f394771-0f24-49d8-8c98-680a13c298b1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:08.574207 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0f394771-0f24-49d8-8c98-680a13c298b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:08.599374 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f394771-0f24-49d8-8c98-680a13c298b1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:08.695993 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f83c96-5dde-4510-b55e-c94ec4b36cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:08.696689 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f83c96-5dde-4510-b55e-c94ec4b36cb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:08.696976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f83c96-5dde-4510-b55e-c94ec4b36cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:08.697351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f83c96-5dde-4510-b55e-c94ec4b36cb4 admin admin] PATH_INFO: `/v3/projects/e02d6af390084c4f98dabd459da5a1da` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:08.698623 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f83c96-5dde-4510-b55e-c94ec4b36cb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e02d6af390084c4f98dabd459da5a1da)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:08.701161 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f83c96-5dde-4510-b55e-c94ec4b36cb4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:08.751467 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:08.785848 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f7f83c96-5dde-4510-b55e-c94ec4b36cb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e02d6af390084c4f98dabd459da5a1da', 'request_id': 'req-f7f83c96-5dde-4510-b55e-c94ec4b36cb4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:08.794281 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f7f83c96-5dde-4510-b55e-c94ec4b36cb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e02d6af390084c4f98dabd459da5a1da', 'request_id': 'req-f7f83c96-5dde-4510-b55e-c94ec4b36cb4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:08.809335 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f7f83c96-5dde-4510-b55e-c94ec4b36cb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e02d6af390084c4f98dabd459da5a1da', 'request_id': 'req-f7f83c96-5dde-4510-b55e-c94ec4b36cb4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:08.830771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f7f83c96-5dde-4510-b55e-c94ec4b36cb4 admin admin] The token cache is being invalidate because project e02d6af390084c4f98dabd459da5a1da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:01:08.831377 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f7f83c96-5dde-4510-b55e-c94ec4b36cb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:08.832316 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f7f83c96-5dde-4510-b55e-c94ec4b36cb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:08.835602 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 998/1984] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:01:08 2022] DELETE /identity/v3/projects/e02d6af390084c4f98dabd459da5a1da => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:01:08.840881 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b231d6-5922-4c27-b640-21e8a9b2717f None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:08.841465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b231d6-5922-4c27-b640-21e8a9b2717f None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:08.842675 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b231d6-5922-4c27-b640-21e8a9b2717f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:08.869143 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-c0b231d6-5922-4c27-b640-21e8a9b2717f None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:08.870838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c0b231d6-5922-4c27-b640-21e8a9b2717f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:08.973480 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 987/1985] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:01:08.976039 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f6ca2-260a-4839-a99e-5606cea3511c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:08.976541 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f6ca2-260a-4839-a99e-5606cea3511c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:08.976956 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f6ca2-260a-4839-a99e-5606cea3511c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:09.010311 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f44f6ca2-260a-4839-a99e-5606cea3511c None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:09.011816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f44f6ca2-260a-4839-a99e-5606cea3511c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:09.021559 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 999/1986] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:01:09.025925 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b231d6-5922-4c27-b640-21e8a9b2717f None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:09.036798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b231d6-5922-4c27-b640-21e8a9b2717f None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:09.046854 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b75c6-05ca-492d-b46c-b05e8dbc820e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:09.047287 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b75c6-05ca-492d-b46c-b05e8dbc820e admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:09.047912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b75c6-05ca-492d-b46c-b05e8dbc820e admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:09.047912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b75c6-05ca-492d-b46c-b05e8dbc820e admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:09.048813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b75c6-05ca-492d-b46c-b05e8dbc820e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:01:09.049115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b75c6-05ca-492d-b46c-b05e8dbc820e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:09.050850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b75c6-05ca-492d-b46c-b05e8dbc820e admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:09.066480 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 988/1987] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:01:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 05:01:09.071089 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f6ca2-260a-4839-a99e-5606cea3511c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:09.082286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f6ca2-260a-4839-a99e-5606cea3511c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:09.089124 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23685bc-49b5-43d3-a024-756ee36f3f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:09.089897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23685bc-49b5-43d3-a024-756ee36f3f85 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:09.090188 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23685bc-49b5-43d3-a024-756ee36f3f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:09.090541 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23685bc-49b5-43d3-a024-756ee36f3f85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:09.091287 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23685bc-49b5-43d3-a024-756ee36f3f85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:09.093507 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23685bc-49b5-43d3-a024-756ee36f3f85 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:09.135223 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1000/1988] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:01:09 2022] POST /identity/v3/projects => generated 367 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:01:09.136438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b75c6-05ca-492d-b46c-b05e8dbc820e admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:09.149307 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b75c6-05ca-492d-b46c-b05e8dbc820e admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:09.242806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2584b-6abb-4775-a066-072a7f036cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:09.242806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2584b-6abb-4775-a066-072a7f036cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:09.242806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2584b-6abb-4775-a066-072a7f036cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:09.242806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2584b-6abb-4775-a066-072a7f036cb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:09.243118 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a2584b-6abb-4775-a066-072a7f036cb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:09.243118 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a2584b-6abb-4775-a066-072a7f036cb6 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:09.276930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 989/1989] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:01:09 2022] POST /identity/v3/users => generated 284 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:01:09.281290 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23685bc-49b5-43d3-a024-756ee36f3f85 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:09.324728 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23685bc-49b5-43d3-a024-756ee36f3f85 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:09.333031 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1bfd8-9582-497b-9824-38df5ff1976e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:09.333973 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1bfd8-9582-497b-9824-38df5ff1976e admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:09.334478 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1bfd8-9582-497b-9824-38df5ff1976e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:09.335220 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1bfd8-9582-497b-9824-38df5ff1976e admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:09.337072 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e1bfd8-9582-497b-9824-38df5ff1976e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:09.340424 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e1bfd8-9582-497b-9824-38df5ff1976e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:09.360238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1001/1990] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:01:09.363814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2584b-6abb-4775-a066-072a7f036cb6 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:09.394904 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2584b-6abb-4775-a066-072a7f036cb6 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:09.405116 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d36764-a910-4d8d-ae12-81f001e92426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:09.406225 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d36764-a910-4d8d-ae12-81f001e92426 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:09.406742 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d36764-a910-4d8d-ae12-81f001e92426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:09.407238 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d36764-a910-4d8d-ae12-81f001e92426 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:09.408588 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d36764-a910-4d8d-ae12-81f001e92426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:09.411944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d36764-a910-4d8d-ae12-81f001e92426 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:09.425592 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 990/1991] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:01:09.430869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1bfd8-9582-497b-9824-38df5ff1976e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:09.441741 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1bfd8-9582-497b-9824-38df5ff1976e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:09.447328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f89898a-cd74-4ac8-91b2-97136b4fbfd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:09.448228 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89898a-cd74-4ac8-91b2-97136b4fbfd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:09.448530 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89898a-cd74-4ac8-91b2-97136b4fbfd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:09.448858 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89898a-cd74-4ac8-91b2-97136b4fbfd6 admin admin] PATH_INFO: `/v3/projects/47c1709e6f71447881832a20e70746b0/users/b7585a95bd4e48aa9e3d8dce5845e3c4/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:09.467851 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f89898a-cd74-4ac8-91b2-97136b4fbfd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47c1709e6f71447881832a20e70746b0, user_id=b7585a95bd4e48aa9e3d8dce5845e3c4, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:09.470678 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f89898a-cd74-4ac8-91b2-97136b4fbfd6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:09.486842 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1002/1992] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:01:09 2022] PUT /identity/v3/projects/47c1709e6f71447881832a20e70746b0/users/b7585a95bd4e48aa9e3d8dce5845e3c4/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:01:09.499917 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1a487-a33b-44a9-a23a-c4a8b73d1926 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:09.500558 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1a487-a33b-44a9-a23a-c4a8b73d1926 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:09.501003 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1a487-a33b-44a9-a23a-c4a8b73d1926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:09.549189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-98d1a487-a33b-44a9-a23a-c4a8b73d1926 None None] MFA Rules not processed for user `b7585a95bd4e48aa9e3d8dce5845e3c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:09.551066 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-98d1a487-a33b-44a9-a23a-c4a8b73d1926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:09.639927 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:09.654234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 991/1993] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:01:09 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:01:09.671159 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f89898a-cd74-4ac8-91b2-97136b4fbfd6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:09.673101 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0f89898a-cd74-4ac8-91b2-97136b4fbfd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:09.785590 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f89898a-cd74-4ac8-91b2-97136b4fbfd6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:09.789580 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57ac13-218e-4377-b20e-8dd6980c3479 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6946a788d2ef49ff873ca8e81990527e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:09.790296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57ac13-218e-4377-b20e-8dd6980c3479 service glance] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:09.790583 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57ac13-218e-4377-b20e-8dd6980c3479 service glance] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:09.790857 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57ac13-218e-4377-b20e-8dd6980c3479 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:09.801246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e57ac13-218e-4377-b20e-8dd6980c3479 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:09.802824 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e57ac13-218e-4377-b20e-8dd6980c3479 service glance] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:09.814352 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1003/1994] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:01:09 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:01:19.653195 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:19.809983 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=6,keystone:SELECT=76,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:35.251772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1a487-a33b-44a9-a23a-c4a8b73d1926 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:35.253446 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-98d1a487-a33b-44a9-a23a-c4a8b73d1926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:35.324790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1a487-a33b-44a9-a23a-c4a8b73d1926 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:35.329616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242d56e-cb28-4bd9-9337-5b6008b79b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:35.330261 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242d56e-cb28-4bd9-9337-5b6008b79b45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:35.330604 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242d56e-cb28-4bd9-9337-5b6008b79b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:35.330823 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242d56e-cb28-4bd9-9337-5b6008b79b45 admin admin] PATH_INFO: `/v3/users/61d57ac613b34adea593c68b13578d8d` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:35.332037 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c242d56e-cb28-4bd9-9337-5b6008b79b45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61d57ac613b34adea593c68b13578d8d)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:35.334004 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c242d56e-cb28-4bd9-9337-5b6008b79b45 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:35.395756 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c242d56e-cb28-4bd9-9337-5b6008b79b45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61d57ac613b34adea593c68b13578d8d'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:35.404094 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c242d56e-cb28-4bd9-9337-5b6008b79b45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61d57ac613b34adea593c68b13578d8d'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:35.414036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c242d56e-cb28-4bd9-9337-5b6008b79b45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61d57ac613b34adea593c68b13578d8d'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:35.414840 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c242d56e-cb28-4bd9-9337-5b6008b79b45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61d57ac613b34adea593c68b13578d8d'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:35.415724 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c242d56e-cb28-4bd9-9337-5b6008b79b45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61d57ac613b34adea593c68b13578d8d'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:35.435646 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 992/1995] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:01:35 2022] DELETE /identity/v3/users/61d57ac613b34adea593c68b13578d8d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:01:35.440363 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6033408-88c5-4d05-a2b5-aa06e604a53e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:35.440924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6033408-88c5-4d05-a2b5-aa06e604a53e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:35.440924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6033408-88c5-4d05-a2b5-aa06e604a53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:35.460981 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b6033408-88c5-4d05-a2b5-aa06e604a53e None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:35.462462 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b6033408-88c5-4d05-a2b5-aa06e604a53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:35.531320 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1004/1996] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:01:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:01:35.541615 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242d56e-cb28-4bd9-9337-5b6008b79b45 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:35.543385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c242d56e-cb28-4bd9-9337-5b6008b79b45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:35.620067 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242d56e-cb28-4bd9-9337-5b6008b79b45 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:35.624812 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec8198-9bc8-4673-af41-8f43a825b90b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:35.625461 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec8198-9bc8-4673-af41-8f43a825b90b service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:35.625749 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec8198-9bc8-4673-af41-8f43a825b90b service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:35.626030 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec8198-9bc8-4673-af41-8f43a825b90b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:35.638369 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec8198-9bc8-4673-af41-8f43a825b90b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:35.640294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec8198-9bc8-4673-af41-8f43a825b90b service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:35.652976 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 993/1997] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:01:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:01:35.823994 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6033408-88c5-4d05-a2b5-aa06e604a53e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:35.826438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b6033408-88c5-4d05-a2b5-aa06e604a53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:35.845717 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6033408-88c5-4d05-a2b5-aa06e604a53e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:35.852594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650fcdb-ce87-4131-94fb-6dc15885b63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:35.853704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650fcdb-ce87-4131-94fb-6dc15885b63f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:35.854170 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650fcdb-ce87-4131-94fb-6dc15885b63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:35.854660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650fcdb-ce87-4131-94fb-6dc15885b63f admin admin] PATH_INFO: `/v3/users/3a37ce859cd541878c8071a85b50eab9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:35.856678 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e650fcdb-ce87-4131-94fb-6dc15885b63f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a37ce859cd541878c8071a85b50eab9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:35.858886 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e650fcdb-ce87-4131-94fb-6dc15885b63f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:35.945564 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e650fcdb-ce87-4131-94fb-6dc15885b63f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a37ce859cd541878c8071a85b50eab9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:35.960965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e650fcdb-ce87-4131-94fb-6dc15885b63f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a37ce859cd541878c8071a85b50eab9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:35.964263 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e650fcdb-ce87-4131-94fb-6dc15885b63f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a37ce859cd541878c8071a85b50eab9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:35.981287 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e650fcdb-ce87-4131-94fb-6dc15885b63f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a37ce859cd541878c8071a85b50eab9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:35.981953 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e650fcdb-ce87-4131-94fb-6dc15885b63f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a37ce859cd541878c8071a85b50eab9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:36.009441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec8198-9bc8-4673-af41-8f43a825b90b service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:36.010582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1005/1998] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:01:35 2022] DELETE /identity/v3/users/3a37ce859cd541878c8071a85b50eab9 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:01:36.012099 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a5ec8198-9bc8-4673-af41-8f43a825b90b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:36.012836 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84db04-dfce-4c6c-8ad8-e92f59e93882 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:36.013123 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84db04-dfce-4c6c-8ad8-e92f59e93882 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:36.013402 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84db04-dfce-4c6c-8ad8-e92f59e93882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:36.036077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7b84db04-dfce-4c6c-8ad8-e92f59e93882 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:36.037112 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7b84db04-dfce-4c6c-8ad8-e92f59e93882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:36.052292 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec8198-9bc8-4673-af41-8f43a825b90b service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:36.082298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:36.124828 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969745cd-0c30-496b-944f-85eec80f7696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:36.125036 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1006/1999] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:01:36.128787 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969745cd-0c30-496b-944f-85eec80f7696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:36.129089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969745cd-0c30-496b-944f-85eec80f7696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:36.129351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969745cd-0c30-496b-944f-85eec80f7696 admin admin] PATH_INFO: `/v3/projects/54ad2fdbdee748cba641763fd7ccd878` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:36.130608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969745cd-0c30-496b-944f-85eec80f7696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54ad2fdbdee748cba641763fd7ccd878)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:36.132606 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969745cd-0c30-496b-944f-85eec80f7696 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:36.136672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84db04-dfce-4c6c-8ad8-e92f59e93882 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:36.138157 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7b84db04-dfce-4c6c-8ad8-e92f59e93882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:36.153356 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:36.188622 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-969745cd-0c30-496b-944f-85eec80f7696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54ad2fdbdee748cba641763fd7ccd878', 'request_id': 'req-969745cd-0c30-496b-944f-85eec80f7696'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:36.199060 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-969745cd-0c30-496b-944f-85eec80f7696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54ad2fdbdee748cba641763fd7ccd878', 'request_id': 'req-969745cd-0c30-496b-944f-85eec80f7696'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:36.203929 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84db04-dfce-4c6c-8ad8-e92f59e93882 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:36.206498 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-969745cd-0c30-496b-944f-85eec80f7696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54ad2fdbdee748cba641763fd7ccd878', 'request_id': 'req-969745cd-0c30-496b-944f-85eec80f7696'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:36.218115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-969745cd-0c30-496b-944f-85eec80f7696 admin admin] The token cache is being invalidate because project 54ad2fdbdee748cba641763fd7ccd878 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:01:36.218432 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-969745cd-0c30-496b-944f-85eec80f7696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:36.219092 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-969745cd-0c30-496b-944f-85eec80f7696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:36.221038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 994/2000] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:01:36 2022] DELETE /identity/v3/projects/54ad2fdbdee748cba641763fd7ccd878 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:01:36.229157 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6480640-0cae-422a-b8e8-55aca4f89f40 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:36.229444 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6480640-0cae-422a-b8e8-55aca4f89f40 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:36.229727 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6480640-0cae-422a-b8e8-55aca4f89f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:36.250916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-a6480640-0cae-422a-b8e8-55aca4f89f40 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:36.252189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a6480640-0cae-422a-b8e8-55aca4f89f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:36.257391 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ee359-ff06-48cb-a149-d6302926a6fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:36.258076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ee359-ff06-48cb-a149-d6302926a6fe service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:36.258366 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ee359-ff06-48cb-a149-d6302926a6fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:36.258643 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ee359-ff06-48cb-a149-d6302926a6fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:36.260423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1f1ee359-ff06-48cb-a149-d6302926a6fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:36.310294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 995/2001] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:01:36.318425 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35132703-8063-4733-9853-b500b9c56797 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:36.318701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35132703-8063-4733-9853-b500b9c56797 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:36.318982 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35132703-8063-4733-9853-b500b9c56797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:36.323443 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ee359-ff06-48cb-a149-d6302926a6fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:36.325364 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ee359-ff06-48cb-a149-d6302926a6fe service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:36.342073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-35132703-8063-4733-9853-b500b9c56797 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:36.343310 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-35132703-8063-4733-9853-b500b9c56797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:36.345154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1007/2002] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:01:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:01:36.351359 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 996/2003] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:01:36.355661 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ee359-ff06-48cb-a149-d6302926a6fe service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:36.365275 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ee359-ff06-48cb-a149-d6302926a6fe service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:36.370051 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2442ec-331f-4de7-94cc-13debe89e411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:36.370686 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2442ec-331f-4de7-94cc-13debe89e411 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:36.370969 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2442ec-331f-4de7-94cc-13debe89e411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:36.371245 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2442ec-331f-4de7-94cc-13debe89e411 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:36.371980 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2442ec-331f-4de7-94cc-13debe89e411 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:01:36.372300 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2442ec-331f-4de7-94cc-13debe89e411 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:36.373982 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2442ec-331f-4de7-94cc-13debe89e411 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:36.387429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1008/2004] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 05:01:36.391514 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35132703-8063-4733-9853-b500b9c56797 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:36.402082 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35132703-8063-4733-9853-b500b9c56797 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:36.410379 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e009731-e53b-4489-b2de-73aa712c8691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:36.411272 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e009731-e53b-4489-b2de-73aa712c8691 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:36.411573 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e009731-e53b-4489-b2de-73aa712c8691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:36.411921 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e009731-e53b-4489-b2de-73aa712c8691 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:36.412956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e009731-e53b-4489-b2de-73aa712c8691 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:36.415930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e009731-e53b-4489-b2de-73aa712c8691 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:36.447180 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 997/2005] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:01:36 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:01:36.451853 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2442ec-331f-4de7-94cc-13debe89e411 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:36.463283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2442ec-331f-4de7-94cc-13debe89e411 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:36.530214 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b56a2-82ea-4bc7-8f29-ec0ff1556621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:36.530828 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b56a2-82ea-4bc7-8f29-ec0ff1556621 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:36.531534 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b56a2-82ea-4bc7-8f29-ec0ff1556621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:36.531534 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b56a2-82ea-4bc7-8f29-ec0ff1556621 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:36.532070 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b56a2-82ea-4bc7-8f29-ec0ff1556621 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:36.534199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b56a2-82ea-4bc7-8f29-ec0ff1556621 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:36.546002 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:36.561847 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1009/2006] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:01:36 2022] POST /identity/v3/users => generated 286 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:01:36.563045 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e009731-e53b-4489-b2de-73aa712c8691 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:36.574232 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e009731-e53b-4489-b2de-73aa712c8691 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:36.579774 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d049b-3555-4f02-bb65-339b37a99349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:36.580418 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d049b-3555-4f02-bb65-339b37a99349 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:36.580708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d049b-3555-4f02-bb65-339b37a99349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:36.580983 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d049b-3555-4f02-bb65-339b37a99349 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:36.581695 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3d049b-3555-4f02-bb65-339b37a99349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:36.583331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3d049b-3555-4f02-bb65-339b37a99349 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:36.593863 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 998/2007] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:01:36.597888 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b56a2-82ea-4bc7-8f29-ec0ff1556621 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:36.608179 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b56a2-82ea-4bc7-8f29-ec0ff1556621 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:36.614433 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f39cd0e-c6cb-46f7-b82d-8878c83229c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:36.615236 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f39cd0e-c6cb-46f7-b82d-8878c83229c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:36.615519 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f39cd0e-c6cb-46f7-b82d-8878c83229c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:36.615886 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f39cd0e-c6cb-46f7-b82d-8878c83229c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:36.616843 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f39cd0e-c6cb-46f7-b82d-8878c83229c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:36.618709 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f39cd0e-c6cb-46f7-b82d-8878c83229c9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:36.629866 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1010/2008] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:01:36.634613 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d049b-3555-4f02-bb65-339b37a99349 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:36.645862 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d049b-3555-4f02-bb65-339b37a99349 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:36.647089 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f39cd0e-c6cb-46f7-b82d-8878c83229c9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:36.649101 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6f39cd0e-c6cb-46f7-b82d-8878c83229c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:36.651803 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75355bee-ff9c-451d-9573-fa72c7241ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:36.652519 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75355bee-ff9c-451d-9573-fa72c7241ca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:36.652797 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75355bee-ff9c-451d-9573-fa72c7241ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:36.653068 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75355bee-ff9c-451d-9573-fa72c7241ca1 admin admin] PATH_INFO: `/v3/projects/8fe56fc2f69d43179f5c8175befbe1e0/users/04da8d850b3a47868513fd82f48937b5/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:36.664205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f39cd0e-c6cb-46f7-b82d-8878c83229c9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:36.666386 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:36.670699 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75355bee-ff9c-451d-9573-fa72c7241ca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fe56fc2f69d43179f5c8175befbe1e0, user_id=04da8d850b3a47868513fd82f48937b5, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:36.677520 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75355bee-ff9c-451d-9573-fa72c7241ca1 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:36.681107 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba48d0-7126-42a3-aabd-3fc90be3d670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:36.681962 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba48d0-7126-42a3-aabd-3fc90be3d670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:36.682445 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba48d0-7126-42a3-aabd-3fc90be3d670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:36.683229 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba48d0-7126-42a3-aabd-3fc90be3d670 admin admin] PATH_INFO: `/v3/projects/bf3d2754f23e496bb7b2f2531297380d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:36.685512 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba48d0-7126-42a3-aabd-3fc90be3d670 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf3d2754f23e496bb7b2f2531297380d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:36.688166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba48d0-7126-42a3-aabd-3fc90be3d670 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:36.694711 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 999/2009] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:01:36 2022] PUT /identity/v3/projects/8fe56fc2f69d43179f5c8175befbe1e0/users/04da8d850b3a47868513fd82f48937b5/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:01:36.706304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00764642-cd6f-4f1b-a981-ea74c9966c5e None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:36.706602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00764642-cd6f-4f1b-a981-ea74c9966c5e None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:36.706885 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00764642-cd6f-4f1b-a981-ea74c9966c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:36.745491 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-00764642-cd6f-4f1b-a981-ea74c9966c5e None None] MFA Rules not processed for user `04da8d850b3a47868513fd82f48937b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:36.746827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-00764642-cd6f-4f1b-a981-ea74c9966c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:36.747719 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9bba48d0-7126-42a3-aabd-3fc90be3d670 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf3d2754f23e496bb7b2f2531297380d', 'request_id': 'req-9bba48d0-7126-42a3-aabd-3fc90be3d670'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:36.754964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9bba48d0-7126-42a3-aabd-3fc90be3d670 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf3d2754f23e496bb7b2f2531297380d', 'request_id': 'req-9bba48d0-7126-42a3-aabd-3fc90be3d670'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:36.764162 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9bba48d0-7126-42a3-aabd-3fc90be3d670 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf3d2754f23e496bb7b2f2531297380d', 'request_id': 'req-9bba48d0-7126-42a3-aabd-3fc90be3d670'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:36.785025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9bba48d0-7126-42a3-aabd-3fc90be3d670 admin admin] The token cache is being invalidate because project bf3d2754f23e496bb7b2f2531297380d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:01:36.785350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9bba48d0-7126-42a3-aabd-3fc90be3d670 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:36.787961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9bba48d0-7126-42a3-aabd-3fc90be3d670 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:36.790206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1011/2010] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:01:36 2022] DELETE /identity/v3/projects/bf3d2754f23e496bb7b2f2531297380d => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:01:36.794294 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba48d0-7126-42a3-aabd-3fc90be3d670 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:36.795912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9bba48d0-7126-42a3-aabd-3fc90be3d670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:36.836501 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1000/2011] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:01:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:01:36.850195 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00764642-cd6f-4f1b-a981-ea74c9966c5e None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:36.852265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-00764642-cd6f-4f1b-a981-ea74c9966c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:36.866998 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba48d0-7126-42a3-aabd-3fc90be3d670 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:36.872199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a253c07-3ab4-49d3-9eb1-5821e9c592b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:36.873001 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a253c07-3ab4-49d3-9eb1-5821e9c592b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:36.873378 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a253c07-3ab4-49d3-9eb1-5821e9c592b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:36.873760 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a253c07-3ab4-49d3-9eb1-5821e9c592b9 admin admin] PATH_INFO: `/v3/users/a1eae282478c434ebbe002513727d47b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:36.875249 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a253c07-3ab4-49d3-9eb1-5821e9c592b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1eae282478c434ebbe002513727d47b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:36.877296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a253c07-3ab4-49d3-9eb1-5821e9c592b9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:36.922352 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00764642-cd6f-4f1b-a981-ea74c9966c5e None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:36.927094 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa0be2-2bc5-4e0a-b9f1-4f5caad25cc9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6946a788d2ef49ff873ca8e81990527e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:36.928022 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa0be2-2bc5-4e0a-b9f1-4f5caad25cc9 service glance] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:36.928554 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa0be2-2bc5-4e0a-b9f1-4f5caad25cc9 service glance] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:36.928961 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa0be2-2bc5-4e0a-b9f1-4f5caad25cc9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:36.938192 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1a253c07-3ab4-49d3-9eb1-5821e9c592b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1eae282478c434ebbe002513727d47b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:36.941532 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa0be2-2bc5-4e0a-b9f1-4f5caad25cc9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:36.943099 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa0be2-2bc5-4e0a-b9f1-4f5caad25cc9 service glance] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:36.945904 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1a253c07-3ab4-49d3-9eb1-5821e9c592b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1eae282478c434ebbe002513727d47b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:36.946650 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1a253c07-3ab4-49d3-9eb1-5821e9c592b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1eae282478c434ebbe002513727d47b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:36.953442 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1001/2012] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:01:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:01:36.956373 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1a253c07-3ab4-49d3-9eb1-5821e9c592b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1eae282478c434ebbe002513727d47b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:36.957021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1a253c07-3ab4-49d3-9eb1-5821e9c592b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1eae282478c434ebbe002513727d47b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:36.964092 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:36.979592 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1012/2013] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:01:36 2022] DELETE /identity/v3/users/a1eae282478c434ebbe002513727d47b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:01:37.323661 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa0be2-2bc5-4e0a-b9f1-4f5caad25cc9 service glance] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:37.351333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa0be2-2bc5-4e0a-b9f1-4f5caad25cc9 service glance] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:37.424713 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3864976-5ddf-44f0-8190-83b846417d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:37.425661 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3864976-5ddf-44f0-8190-83b846417d3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:37.425848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3864976-5ddf-44f0-8190-83b846417d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:37.426191 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3864976-5ddf-44f0-8190-83b846417d3e admin admin] PATH_INFO: `/v3/projects/ef2dac306deb47d19df60262012c7fb8` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:37.427614 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3864976-5ddf-44f0-8190-83b846417d3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef2dac306deb47d19df60262012c7fb8)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:37.429721 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3864976-5ddf-44f0-8190-83b846417d3e admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:37.461792 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:37.505178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-d3864976-5ddf-44f0-8190-83b846417d3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef2dac306deb47d19df60262012c7fb8', 'request_id': 'req-d3864976-5ddf-44f0-8190-83b846417d3e'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:37.518344 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-d3864976-5ddf-44f0-8190-83b846417d3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef2dac306deb47d19df60262012c7fb8', 'request_id': 'req-d3864976-5ddf-44f0-8190-83b846417d3e'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:37.536213 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-d3864976-5ddf-44f0-8190-83b846417d3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef2dac306deb47d19df60262012c7fb8', 'request_id': 'req-d3864976-5ddf-44f0-8190-83b846417d3e'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:37.551704 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-d3864976-5ddf-44f0-8190-83b846417d3e admin admin] The token cache is being invalidate because project ef2dac306deb47d19df60262012c7fb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:01:37.552275 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-d3864976-5ddf-44f0-8190-83b846417d3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:37.553105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-d3864976-5ddf-44f0-8190-83b846417d3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:37.556114 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1002/2014] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:01:37 2022] DELETE /identity/v3/projects/ef2dac306deb47d19df60262012c7fb8 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:01:37.561912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16af9f1-53fa-4052-8138-66fcaeabdb86 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:37.562429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16af9f1-53fa-4052-8138-66fcaeabdb86 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:37.562515 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16af9f1-53fa-4052-8138-66fcaeabdb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:37.589875 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c16af9f1-53fa-4052-8138-66fcaeabdb86 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:37.591113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c16af9f1-53fa-4052-8138-66fcaeabdb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:37.689681 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1013/2015] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:01:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:01:37.697690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551d107-fd51-4176-9319-cd754edfdf39 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:37.698222 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551d107-fd51-4176-9319-cd754edfdf39 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:37.698687 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551d107-fd51-4176-9319-cd754edfdf39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:37.723633 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-4551d107-fd51-4176-9319-cd754edfdf39 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:37.724957 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-4551d107-fd51-4176-9319-cd754edfdf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:37.733200 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1003/2016] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:01:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:01:37.737666 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16af9f1-53fa-4052-8138-66fcaeabdb86 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:37.752448 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16af9f1-53fa-4052-8138-66fcaeabdb86 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:37.758045 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7301ad-3d78-4268-93ca-4043cc443708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:37.758685 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7301ad-3d78-4268-93ca-4043cc443708 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:37.759008 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7301ad-3d78-4268-93ca-4043cc443708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:37.759289 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7301ad-3d78-4268-93ca-4043cc443708 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:37.760063 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7301ad-3d78-4268-93ca-4043cc443708 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:01:37.760384 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7301ad-3d78-4268-93ca-4043cc443708 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:37.762039 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7301ad-3d78-4268-93ca-4043cc443708 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:37.773909 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1014/2017] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 05:01:37.778457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551d107-fd51-4176-9319-cd754edfdf39 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:37.787968 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551d107-fd51-4176-9319-cd754edfdf39 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:37.793011 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449cf8d-9571-481f-b053-269842ee0578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:37.793602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449cf8d-9571-481f-b053-269842ee0578 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:37.793883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449cf8d-9571-481f-b053-269842ee0578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:37.794163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449cf8d-9571-481f-b053-269842ee0578 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:37.794825 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449cf8d-9571-481f-b053-269842ee0578 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:37.796797 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449cf8d-9571-481f-b053-269842ee0578 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:37.825258 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1004/2018] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:01:37 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:01:37.829790 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7301ad-3d78-4268-93ca-4043cc443708 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:37.840175 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7301ad-3d78-4268-93ca-4043cc443708 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:37.910020 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a920e2-fda7-44a3-a724-1a1e1f4df962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:37.910570 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a920e2-fda7-44a3-a724-1a1e1f4df962 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:37.910858 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a920e2-fda7-44a3-a724-1a1e1f4df962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:37.911149 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a920e2-fda7-44a3-a724-1a1e1f4df962 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:37.911845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a920e2-fda7-44a3-a724-1a1e1f4df962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:37.913783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a920e2-fda7-44a3-a724-1a1e1f4df962 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:37.937614 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1015/2019] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:01:37 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:01:37.942194 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449cf8d-9571-481f-b053-269842ee0578 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:37.954012 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449cf8d-9571-481f-b053-269842ee0578 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:37.959010 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e3e3b-5337-4a7b-b9f3-c0a7ef618a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:37.959671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e3e3b-5337-4a7b-b9f3-c0a7ef618a4e admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:37.959956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e3e3b-5337-4a7b-b9f3-c0a7ef618a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:37.960232 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e3e3b-5337-4a7b-b9f3-c0a7ef618a4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:37.960936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e3e3b-5337-4a7b-b9f3-c0a7ef618a4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:37.962624 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e3e3b-5337-4a7b-b9f3-c0a7ef618a4e admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:37.973630 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1005/2020] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:01:37.979705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a920e2-fda7-44a3-a724-1a1e1f4df962 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:37.996742 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a920e2-fda7-44a3-a724-1a1e1f4df962 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:38.002092 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb389a1-b1b4-4752-87b7-d5de6c455dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:38.002747 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb389a1-b1b4-4752-87b7-d5de6c455dff admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:38.003034 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb389a1-b1b4-4752-87b7-d5de6c455dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:38.003315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb389a1-b1b4-4752-87b7-d5de6c455dff admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:38.004067 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb389a1-b1b4-4752-87b7-d5de6c455dff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:38.005709 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb389a1-b1b4-4752-87b7-d5de6c455dff admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:38.018011 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1016/2021] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:01:38.022238 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e3e3b-5337-4a7b-b9f3-c0a7ef618a4e admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:38.033398 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e3e3b-5337-4a7b-b9f3-c0a7ef618a4e admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:38.038901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352eb33-3ecd-49b8-803f-82fb16517359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:38.040307 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352eb33-3ecd-49b8-803f-82fb16517359 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:38.040617 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352eb33-3ecd-49b8-803f-82fb16517359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:38.040930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352eb33-3ecd-49b8-803f-82fb16517359 admin admin] PATH_INFO: `/v3/projects/58d3971195424fd7a5eee7776c88c42c/users/172d92b1edf94ee6895264f700b33597/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:38.060552 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9352eb33-3ecd-49b8-803f-82fb16517359 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58d3971195424fd7a5eee7776c88c42c, user_id=172d92b1edf94ee6895264f700b33597, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:38.063317 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9352eb33-3ecd-49b8-803f-82fb16517359 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:38.080839 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1006/2022] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:01:38 2022] PUT /identity/v3/projects/58d3971195424fd7a5eee7776c88c42c/users/172d92b1edf94ee6895264f700b33597/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:01:38.091588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270af21-9eab-499a-a753-8f249f1132ab None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:38.091890 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270af21-9eab-499a-a753-8f249f1132ab None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:38.092202 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270af21-9eab-499a-a753-8f249f1132ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:38.123990 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5270af21-9eab-499a-a753-8f249f1132ab None None] MFA Rules not processed for user `172d92b1edf94ee6895264f700b33597`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:38.125493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5270af21-9eab-499a-a753-8f249f1132ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:38.186383 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:38.217071 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1017/2023] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:01:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:01:38.221880 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352eb33-3ecd-49b8-803f-82fb16517359 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:38.296344 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352eb33-3ecd-49b8-803f-82fb16517359 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:38.301777 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b61c9-ba05-484c-949d-1415f9813637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:38.302422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b61c9-ba05-484c-949d-1415f9813637 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:38.302705 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b61c9-ba05-484c-949d-1415f9813637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:38.302980 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b61c9-ba05-484c-949d-1415f9813637 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:38.303702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b61c9-ba05-484c-949d-1415f9813637 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:38.305882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b61c9-ba05-484c-949d-1415f9813637 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:38.345865 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1007/2024] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:01:38 2022] POST /identity/v3/projects => generated 379 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:01:38.351001 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270af21-9eab-499a-a753-8f249f1132ab None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:38.362274 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270af21-9eab-499a-a753-8f249f1132ab None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:38.429444 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4664ed-ae58-476d-a093-f844a2e62200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:38.430180 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4664ed-ae58-476d-a093-f844a2e62200 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:38.430460 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4664ed-ae58-476d-a093-f844a2e62200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:38.430743 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4664ed-ae58-476d-a093-f844a2e62200 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:38.431401 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4664ed-ae58-476d-a093-f844a2e62200 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:38.433471 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4664ed-ae58-476d-a093-f844a2e62200 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:38.462218 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1018/2025] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:01:38 2022] POST /identity/v3/users => generated 296 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:01:38.466455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b61c9-ba05-484c-949d-1415f9813637 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:38.477363 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b61c9-ba05-484c-949d-1415f9813637 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:38.481951 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0729d-d038-407b-bbc6-ce838b3b5a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:38.482578 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0729d-d038-407b-bbc6-ce838b3b5a7e admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:38.482866 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0729d-d038-407b-bbc6-ce838b3b5a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:38.483144 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0729d-d038-407b-bbc6-ce838b3b5a7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:38.483884 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0729d-d038-407b-bbc6-ce838b3b5a7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:38.485501 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0729d-d038-407b-bbc6-ce838b3b5a7e admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:38.501048 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1008/2026] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:01:38.502234 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4664ed-ae58-476d-a093-f844a2e62200 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:38.514219 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4664ed-ae58-476d-a093-f844a2e62200 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:38.520136 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c884a-c039-45fe-9dc0-f10221175453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:38.520823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c884a-c039-45fe-9dc0-f10221175453 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:38.521112 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c884a-c039-45fe-9dc0-f10221175453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:38.521393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c884a-c039-45fe-9dc0-f10221175453 admin admin] PATH_INFO: `/v3/projects/56e7111569344951ac9391ef490bd4a1/users/ac550a65af0c433d97b66a4760e29a05/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:38.546360 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c884a-c039-45fe-9dc0-f10221175453 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56e7111569344951ac9391ef490bd4a1, user_id=ac550a65af0c433d97b66a4760e29a05, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:38.549132 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c884a-c039-45fe-9dc0-f10221175453 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:38.563121 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1019/2027] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:01:38 2022] PUT /identity/v3/projects/56e7111569344951ac9391ef490bd4a1/users/ac550a65af0c433d97b66a4760e29a05/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:01:38.567345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0729d-d038-407b-bbc6-ce838b3b5a7e admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:38.578618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0729d-d038-407b-bbc6-ce838b3b5a7e admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:38.597746 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:38.646140 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a93ac-8c87-43d2-b1ae-30fa4841ccac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:38.646786 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a93ac-8c87-43d2-b1ae-30fa4841ccac admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:38.647076 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a93ac-8c87-43d2-b1ae-30fa4841ccac admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:38.647355 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a93ac-8c87-43d2-b1ae-30fa4841ccac admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:38.648121 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a93ac-8c87-43d2-b1ae-30fa4841ccac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:38.649822 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a93ac-8c87-43d2-b1ae-30fa4841ccac admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:38.664793 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1009/2028] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:01:38.669419 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c884a-c039-45fe-9dc0-f10221175453 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:38.681099 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c884a-c039-45fe-9dc0-f10221175453 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:38.686788 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6768497f-2fb7-4437-97dc-fb7e472eae10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:38.688028 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768497f-2fb7-4437-97dc-fb7e472eae10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:38.688526 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768497f-2fb7-4437-97dc-fb7e472eae10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:38.688875 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768497f-2fb7-4437-97dc-fb7e472eae10 admin admin] PATH_INFO: `/v3/projects/56e7111569344951ac9391ef490bd4a1/users/ac550a65af0c433d97b66a4760e29a05/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:38.691457 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6768497f-2fb7-4437-97dc-fb7e472eae10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56e7111569344951ac9391ef490bd4a1, user_id=ac550a65af0c433d97b66a4760e29a05, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:38.694247 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6768497f-2fb7-4437-97dc-fb7e472eae10 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:38.710107 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1020/2029] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:01:38 2022] PUT /identity/v3/projects/56e7111569344951ac9391ef490bd4a1/users/ac550a65af0c433d97b66a4760e29a05/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:01:38.721972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc87eb-4e50-460f-87a5-c218beac8eb7 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:38.722352 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc87eb-4e50-460f-87a5-c218beac8eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:38.722827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc87eb-4e50-460f-87a5-c218beac8eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:38.751531 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-2bdc87eb-4e50-460f-87a5-c218beac8eb7 None None] MFA Rules not processed for user `ac550a65af0c433d97b66a4760e29a05`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:38.752790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-2bdc87eb-4e50-460f-87a5-c218beac8eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:38.824765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1010/2030] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:01:38 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:01:38.837653 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6768497f-2fb7-4437-97dc-fb7e472eae10 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:38.839716 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6768497f-2fb7-4437-97dc-fb7e472eae10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:38.914043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6768497f-2fb7-4437-97dc-fb7e472eae10 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:38.917833 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9dab9-fc0c-4b19-b582-6dbaeea1affb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:38.918473 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9dab9-fc0c-4b19-b582-6dbaeea1affb service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:38.918760 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9dab9-fc0c-4b19-b582-6dbaeea1affb service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:38.919034 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9dab9-fc0c-4b19-b582-6dbaeea1affb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:38.930168 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9dab9-fc0c-4b19-b582-6dbaeea1affb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:38.931675 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9dab9-fc0c-4b19-b582-6dbaeea1affb service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:38.941929 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1021/2031] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:01:38 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:01:39.041491 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdc87eb-4e50-460f-87a5-c218beac8eb7 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:39.051424 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdc87eb-4e50-460f-87a5-c218beac8eb7 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:39.055816 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b7415e-df39-418d-b184-a02cf0aea68c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:39.056517 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7415e-df39-418d-b184-a02cf0aea68c service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:39.056805 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7415e-df39-418d-b184-a02cf0aea68c service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:39.057077 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7415e-df39-418d-b184-a02cf0aea68c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:39.125031 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b7415e-df39-418d-b184-a02cf0aea68c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:39.127046 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b7415e-df39-418d-b184-a02cf0aea68c service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:39.136595 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1011/2032] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:01:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:01:39.431792 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9dab9-fc0c-4b19-b582-6dbaeea1affb service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:39.478050 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:39.506247 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9dab9-fc0c-4b19-b582-6dbaeea1affb service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:39.512032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa4674-edfd-44c9-9028-828c0b63396a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:39.512945 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa4674-edfd-44c9-9028-828c0b63396a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:39.513210 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa4674-edfd-44c9-9028-828c0b63396a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:39.513514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa4674-edfd-44c9-9028-828c0b63396a admin admin] PATH_INFO: `/v3/users/172d92b1edf94ee6895264f700b33597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:39.514763 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa4674-edfd-44c9-9028-828c0b63396a admin admin] RBAC: Authorizing `identity:delete_user(user_id=172d92b1edf94ee6895264f700b33597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:39.517691 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa4674-edfd-44c9-9028-828c0b63396a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:39.582892 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b6aa4674-edfd-44c9-9028-828c0b63396a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '172d92b1edf94ee6895264f700b33597'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:39.591462 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b6aa4674-edfd-44c9-9028-828c0b63396a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '172d92b1edf94ee6895264f700b33597'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:39.592422 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b6aa4674-edfd-44c9-9028-828c0b63396a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '172d92b1edf94ee6895264f700b33597'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:39.603853 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b6aa4674-edfd-44c9-9028-828c0b63396a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '172d92b1edf94ee6895264f700b33597'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:39.604678 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b6aa4674-edfd-44c9-9028-828c0b63396a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '172d92b1edf94ee6895264f700b33597'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:39.625185 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1022/2033] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:01:39 2022] DELETE /identity/v3/users/172d92b1edf94ee6895264f700b33597 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:01:39.629690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14fa91-5ae4-4eed-805f-05b626c40353 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:39.630121 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14fa91-5ae4-4eed-805f-05b626c40353 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:39.630296 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14fa91-5ae4-4eed-805f-05b626c40353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:39.653382 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-0c14fa91-5ae4-4eed-805f-05b626c40353 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:39.654628 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-0c14fa91-5ae4-4eed-805f-05b626c40353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:39.705382 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:39.731672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1012/2034] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:01:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:01:39.741948 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa4674-edfd-44c9-9028-828c0b63396a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:39.743490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b6aa4674-edfd-44c9-9028-828c0b63396a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:39.809765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa4674-edfd-44c9-9028-828c0b63396a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:39.813796 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8747d0-55c1-4b83-a4e2-3dd36f7ba9b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:39.814595 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8747d0-55c1-4b83-a4e2-3dd36f7ba9b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:39.814979 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8747d0-55c1-4b83-a4e2-3dd36f7ba9b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:39.815358 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8747d0-55c1-4b83-a4e2-3dd36f7ba9b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:39.826310 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8747d0-55c1-4b83-a4e2-3dd36f7ba9b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:39.828232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8747d0-55c1-4b83-a4e2-3dd36f7ba9b2 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:39.838781 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1023/2035] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:01:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:01:40.132525 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14fa91-5ae4-4eed-805f-05b626c40353 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:40.133558 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-0c14fa91-5ae4-4eed-805f-05b626c40353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:40.147752 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14fa91-5ae4-4eed-805f-05b626c40353 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:40.153229 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f0d04-b972-4940-80e7-ef5dd0fd71af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:40.153865 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f0d04-b972-4940-80e7-ef5dd0fd71af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:40.154101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f0d04-b972-4940-80e7-ef5dd0fd71af admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:40.154427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f0d04-b972-4940-80e7-ef5dd0fd71af admin admin] PATH_INFO: `/v3/projects/58d3971195424fd7a5eee7776c88c42c` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:40.157943 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f0d04-b972-4940-80e7-ef5dd0fd71af admin admin] RBAC: Authorizing `identity:delete_project(project_id=58d3971195424fd7a5eee7776c88c42c)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:40.157943 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f0d04-b972-4940-80e7-ef5dd0fd71af admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:40.207382 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-960f0d04-b972-4940-80e7-ef5dd0fd71af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58d3971195424fd7a5eee7776c88c42c', 'request_id': 'req-960f0d04-b972-4940-80e7-ef5dd0fd71af'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:40.216765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-960f0d04-b972-4940-80e7-ef5dd0fd71af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58d3971195424fd7a5eee7776c88c42c', 'request_id': 'req-960f0d04-b972-4940-80e7-ef5dd0fd71af'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:40.227005 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-960f0d04-b972-4940-80e7-ef5dd0fd71af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58d3971195424fd7a5eee7776c88c42c', 'request_id': 'req-960f0d04-b972-4940-80e7-ef5dd0fd71af'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:40.245948 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-960f0d04-b972-4940-80e7-ef5dd0fd71af admin admin] The token cache is being invalidate because project 58d3971195424fd7a5eee7776c88c42c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:01:40.246242 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-960f0d04-b972-4940-80e7-ef5dd0fd71af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:40.247149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-960f0d04-b972-4940-80e7-ef5dd0fd71af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:40.249759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1013/2036] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:01:40 2022] DELETE /identity/v3/projects/58d3971195424fd7a5eee7776c88c42c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:01:40.254181 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8747d0-55c1-4b83-a4e2-3dd36f7ba9b2 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:40.267508 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8747d0-55c1-4b83-a4e2-3dd36f7ba9b2 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:40.333026 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722871b-44ad-4de3-80b8-f113dbc21247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:40.333879 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722871b-44ad-4de3-80b8-f113dbc21247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:40.334169 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722871b-44ad-4de3-80b8-f113dbc21247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:40.334446 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722871b-44ad-4de3-80b8-f113dbc21247 admin admin] PATH_INFO: `/v3/users/ac550a65af0c433d97b66a4760e29a05` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:40.335720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8722871b-44ad-4de3-80b8-f113dbc21247 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac550a65af0c433d97b66a4760e29a05)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:40.337666 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8722871b-44ad-4de3-80b8-f113dbc21247 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:40.339294 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:40.393147 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8722871b-44ad-4de3-80b8-f113dbc21247 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac550a65af0c433d97b66a4760e29a05'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:40.400294 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8722871b-44ad-4de3-80b8-f113dbc21247 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac550a65af0c433d97b66a4760e29a05'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:40.400998 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8722871b-44ad-4de3-80b8-f113dbc21247 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac550a65af0c433d97b66a4760e29a05'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:40.410239 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8722871b-44ad-4de3-80b8-f113dbc21247 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac550a65af0c433d97b66a4760e29a05'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:40.410239 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8722871b-44ad-4de3-80b8-f113dbc21247 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac550a65af0c433d97b66a4760e29a05'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:40.429047 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1024/2037] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:01:40 2022] DELETE /identity/v3/users/ac550a65af0c433d97b66a4760e29a05 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:01:40.726934 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f0d04-b972-4940-80e7-ef5dd0fd71af admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:40.729453 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-960f0d04-b972-4940-80e7-ef5dd0fd71af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:40.756872 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f0d04-b972-4940-80e7-ef5dd0fd71af admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:40.818079 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed003c3c-6b8b-42ae-abf2-e5f872c56f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:40.818767 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed003c3c-6b8b-42ae-abf2-e5f872c56f2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:40.819051 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed003c3c-6b8b-42ae-abf2-e5f872c56f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:40.819331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed003c3c-6b8b-42ae-abf2-e5f872c56f2f admin admin] PATH_INFO: `/v3/projects/56e7111569344951ac9391ef490bd4a1` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:40.820604 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed003c3c-6b8b-42ae-abf2-e5f872c56f2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=56e7111569344951ac9391ef490bd4a1)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:40.822564 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed003c3c-6b8b-42ae-abf2-e5f872c56f2f admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:40.871331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-ed003c3c-6b8b-42ae-abf2-e5f872c56f2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56e7111569344951ac9391ef490bd4a1', 'request_id': 'req-ed003c3c-6b8b-42ae-abf2-e5f872c56f2f'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:40.879967 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-ed003c3c-6b8b-42ae-abf2-e5f872c56f2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56e7111569344951ac9391ef490bd4a1', 'request_id': 'req-ed003c3c-6b8b-42ae-abf2-e5f872c56f2f'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:40.886715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-ed003c3c-6b8b-42ae-abf2-e5f872c56f2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56e7111569344951ac9391ef490bd4a1', 'request_id': 'req-ed003c3c-6b8b-42ae-abf2-e5f872c56f2f'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:40.891367 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:40.899066 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-ed003c3c-6b8b-42ae-abf2-e5f872c56f2f admin admin] The token cache is being invalidate because project 56e7111569344951ac9391ef490bd4a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:01:40.899362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-ed003c3c-6b8b-42ae-abf2-e5f872c56f2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:40.900287 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-ed003c3c-6b8b-42ae-abf2-e5f872c56f2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:40.902328 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1014/2038] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:01:40 2022] DELETE /identity/v3/projects/56e7111569344951ac9391ef490bd4a1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:01:40.907403 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57082cb3-4508-473d-9b85-5f2d4fe90778 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:40.907891 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57082cb3-4508-473d-9b85-5f2d4fe90778 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:40.907978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57082cb3-4508-473d-9b85-5f2d4fe90778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:40.929513 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-57082cb3-4508-473d-9b85-5f2d4fe90778 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:40.930822 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-57082cb3-4508-473d-9b85-5f2d4fe90778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:40.995294 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1025/2039] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:01:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:01:41.003215 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0423b-349c-402d-9cb0-b0d7efbefaf1 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:41.003502 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0423b-349c-402d-9cb0-b0d7efbefaf1 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:41.003810 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0423b-349c-402d-9cb0-b0d7efbefaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:41.026731 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-22b0423b-349c-402d-9cb0-b0d7efbefaf1 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:41.028025 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-22b0423b-349c-402d-9cb0-b0d7efbefaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:41.036276 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1015/2040] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:01:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:01:41.040911 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57082cb3-4508-473d-9b85-5f2d4fe90778 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:41.050957 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57082cb3-4508-473d-9b85-5f2d4fe90778 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:41.055778 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa358eb8-00e6-430a-8dbc-825b10833832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:41.056389 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa358eb8-00e6-430a-8dbc-825b10833832 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:41.056670 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa358eb8-00e6-430a-8dbc-825b10833832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:41.056945 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa358eb8-00e6-430a-8dbc-825b10833832 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:41.057654 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa358eb8-00e6-430a-8dbc-825b10833832 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:01:41.057959 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa358eb8-00e6-430a-8dbc-825b10833832 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:41.059609 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa358eb8-00e6-430a-8dbc-825b10833832 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:41.071889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1026/2041] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:01:41.077293 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0423b-349c-402d-9cb0-b0d7efbefaf1 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:41.091215 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0423b-349c-402d-9cb0-b0d7efbefaf1 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:41.100351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed6be9-3afb-474e-b932-b864e17a0162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:41.101203 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed6be9-3afb-474e-b932-b864e17a0162 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:41.101597 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed6be9-3afb-474e-b932-b864e17a0162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:41.102040 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed6be9-3afb-474e-b932-b864e17a0162 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:41.102713 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed6be9-3afb-474e-b932-b864e17a0162 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:41.104713 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed6be9-3afb-474e-b932-b864e17a0162 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:41.135982 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1016/2042] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:01:41 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:01:41.140716 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa358eb8-00e6-430a-8dbc-825b10833832 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:41.153383 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa358eb8-00e6-430a-8dbc-825b10833832 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:41.215085 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a342891-1acd-40d3-9478-6ae4ab8c0020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:41.215744 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a342891-1acd-40d3-9478-6ae4ab8c0020 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:41.216041 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a342891-1acd-40d3-9478-6ae4ab8c0020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:41.216321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a342891-1acd-40d3-9478-6ae4ab8c0020 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:41.216994 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a342891-1acd-40d3-9478-6ae4ab8c0020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:41.218957 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a342891-1acd-40d3-9478-6ae4ab8c0020 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:41.231603 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:41.247802 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1027/2043] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:01:41 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:01:41.252330 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed6be9-3afb-474e-b932-b864e17a0162 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:41.263445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed6be9-3afb-474e-b932-b864e17a0162 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:41.269151 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c416a-a12d-4eb2-82d7-1447e8453940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:41.269712 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c416a-a12d-4eb2-82d7-1447e8453940 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:41.269998 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c416a-a12d-4eb2-82d7-1447e8453940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:41.270274 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c416a-a12d-4eb2-82d7-1447e8453940 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:41.271105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c416a-a12d-4eb2-82d7-1447e8453940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:41.272749 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c416a-a12d-4eb2-82d7-1447e8453940 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:41.285085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1017/2044] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:01:41.289547 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a342891-1acd-40d3-9478-6ae4ab8c0020 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:41.302206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a342891-1acd-40d3-9478-6ae4ab8c0020 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:41.307576 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e7b4b-0ce6-4f19-80a8-f80a08820275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:41.308227 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e7b4b-0ce6-4f19-80a8-f80a08820275 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:41.308510 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e7b4b-0ce6-4f19-80a8-f80a08820275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:41.308783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e7b4b-0ce6-4f19-80a8-f80a08820275 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:41.309494 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e7b4b-0ce6-4f19-80a8-f80a08820275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:41.311112 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e7b4b-0ce6-4f19-80a8-f80a08820275 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:41.322581 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1028/2045] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:01:41.327302 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c416a-a12d-4eb2-82d7-1447e8453940 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:41.341353 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c416a-a12d-4eb2-82d7-1447e8453940 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:41.349405 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14b163-083f-4f77-9ddb-d7d3170b7a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:41.350565 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14b163-083f-4f77-9ddb-d7d3170b7a38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:41.351059 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14b163-083f-4f77-9ddb-d7d3170b7a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:41.351579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14b163-083f-4f77-9ddb-d7d3170b7a38 admin admin] PATH_INFO: `/v3/projects/1ae12dd973424019b41445b6f0842f32/users/7ca6673bd7ae41edade8656e34330d12/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:41.373394 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f14b163-083f-4f77-9ddb-d7d3170b7a38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae12dd973424019b41445b6f0842f32, user_id=7ca6673bd7ae41edade8656e34330d12, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:41.376116 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f14b163-083f-4f77-9ddb-d7d3170b7a38 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:41.398633 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1018/2046] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:01:41 2022] PUT /identity/v3/projects/1ae12dd973424019b41445b6f0842f32/users/7ca6673bd7ae41edade8656e34330d12/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:01:41.410170 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f24414-e4b2-492c-be8e-7435a278b76a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:41.410465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f24414-e4b2-492c-be8e-7435a278b76a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:41.410701 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f24414-e4b2-492c-be8e-7435a278b76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:41.442673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-53f24414-e4b2-492c-be8e-7435a278b76a None None] MFA Rules not processed for user `7ca6673bd7ae41edade8656e34330d12`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:41.443940 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-53f24414-e4b2-492c-be8e-7435a278b76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:41.527167 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1029/2047] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:01:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:01:41.531957 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14b163-083f-4f77-9ddb-d7d3170b7a38 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:41.600027 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14b163-083f-4f77-9ddb-d7d3170b7a38 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:41.604941 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f1f54-ef09-4647-a70c-76f053d107af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:41.605575 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f1f54-ef09-4647-a70c-76f053d107af admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:41.605885 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f1f54-ef09-4647-a70c-76f053d107af admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:41.606115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f1f54-ef09-4647-a70c-76f053d107af admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:41.606899 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f1f54-ef09-4647-a70c-76f053d107af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:41.609529 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f1f54-ef09-4647-a70c-76f053d107af admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:41.645068 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1019/2048] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:01:41 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:01:41.646059 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f24414-e4b2-492c-be8e-7435a278b76a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:41.657190 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f24414-e4b2-492c-be8e-7435a278b76a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:41.719392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002d1cd-b2a4-404d-8663-410dafc4c2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:41.720065 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002d1cd-b2a4-404d-8663-410dafc4c2c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:41.720347 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002d1cd-b2a4-404d-8663-410dafc4c2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:41.720615 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002d1cd-b2a4-404d-8663-410dafc4c2c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:41.721290 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002d1cd-b2a4-404d-8663-410dafc4c2c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:41.723222 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002d1cd-b2a4-404d-8663-410dafc4c2c0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:41.751458 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1030/2049] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:01:41 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:01:41.752554 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f1f54-ef09-4647-a70c-76f053d107af admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:41.767200 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f1f54-ef09-4647-a70c-76f053d107af admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:41.772486 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e63de-b9f4-4c44-bc33-f22b1374d992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:41.773216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e63de-b9f4-4c44-bc33-f22b1374d992 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:41.773871 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e63de-b9f4-4c44-bc33-f22b1374d992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:41.774157 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e63de-b9f4-4c44-bc33-f22b1374d992 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:41.774876 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791e63de-b9f4-4c44-bc33-f22b1374d992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:41.776571 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791e63de-b9f4-4c44-bc33-f22b1374d992 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:41.803895 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1020/2050] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:01:41.805182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002d1cd-b2a4-404d-8663-410dafc4c2c0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:41.815929 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002d1cd-b2a4-404d-8663-410dafc4c2c0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:41.829701 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b131a-e7e8-4999-b19f-a164be9d7315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:41.830414 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b131a-e7e8-4999-b19f-a164be9d7315 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:41.830691 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b131a-e7e8-4999-b19f-a164be9d7315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:41.830987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b131a-e7e8-4999-b19f-a164be9d7315 admin admin] PATH_INFO: `/v3/projects/a6f8618139524e5d95685be35349b6da/users/73f8d4f76dc246388576b6c863116b24/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:41.847886 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b131a-e7e8-4999-b19f-a164be9d7315 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6f8618139524e5d95685be35349b6da, user_id=73f8d4f76dc246388576b6c863116b24, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:41.850600 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b131a-e7e8-4999-b19f-a164be9d7315 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:41.868143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1031/2051] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:01:41 2022] PUT /identity/v3/projects/a6f8618139524e5d95685be35349b6da/users/73f8d4f76dc246388576b6c863116b24/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:01:41.872646 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e63de-b9f4-4c44-bc33-f22b1374d992 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:41.883503 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e63de-b9f4-4c44-bc33-f22b1374d992 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:41.949070 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0b5cb-ee18-404c-98d1-be0d83c46fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:41.949984 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0b5cb-ee18-404c-98d1-be0d83c46fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:41.950270 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0b5cb-ee18-404c-98d1-be0d83c46fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:41.950545 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0b5cb-ee18-404c-98d1-be0d83c46fb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:41.951264 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea0b5cb-ee18-404c-98d1-be0d83c46fb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:41.953091 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea0b5cb-ee18-404c-98d1-be0d83c46fb5 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:41.964768 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1021/2052] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:01:41.969317 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b131a-e7e8-4999-b19f-a164be9d7315 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:41.981172 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b131a-e7e8-4999-b19f-a164be9d7315 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:41.988370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794fbad2-1b2b-47d8-96bc-5d87ede48d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:41.989045 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fbad2-1b2b-47d8-96bc-5d87ede48d64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:41.989342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fbad2-1b2b-47d8-96bc-5d87ede48d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:41.989617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fbad2-1b2b-47d8-96bc-5d87ede48d64 admin admin] PATH_INFO: `/v3/projects/a6f8618139524e5d95685be35349b6da/users/73f8d4f76dc246388576b6c863116b24/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:41.991854 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794fbad2-1b2b-47d8-96bc-5d87ede48d64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6f8618139524e5d95685be35349b6da, user_id=73f8d4f76dc246388576b6c863116b24, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:41.994811 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794fbad2-1b2b-47d8-96bc-5d87ede48d64 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:42.006659 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1032/2053] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:01:41 2022] PUT /identity/v3/projects/a6f8618139524e5d95685be35349b6da/users/73f8d4f76dc246388576b6c863116b24/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:01:42.017560 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7e7d1-b2b9-4b74-986e-6bf1e6101fcb None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:42.018017 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7e7d1-b2b9-4b74-986e-6bf1e6101fcb None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:42.018299 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7e7d1-b2b9-4b74-986e-6bf1e6101fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:42.026177 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:42.055142 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-f8c7e7d1-b2b9-4b74-986e-6bf1e6101fcb None None] MFA Rules not processed for user `73f8d4f76dc246388576b6c863116b24`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:42.056777 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-f8c7e7d1-b2b9-4b74-986e-6bf1e6101fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:42.133620 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1022/2054] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:01:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:01:42.144639 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794fbad2-1b2b-47d8-96bc-5d87ede48d64 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:42.146579 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-794fbad2-1b2b-47d8-96bc-5d87ede48d64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:42.179171 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:42.211250 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794fbad2-1b2b-47d8-96bc-5d87ede48d64 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:42.215915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19be6a-261c-4e0a-826e-162f714dd023 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:42.216654 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19be6a-261c-4e0a-826e-162f714dd023 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:42.216932 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19be6a-261c-4e0a-826e-162f714dd023 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:42.217203 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19be6a-261c-4e0a-826e-162f714dd023 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:42.281831 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19be6a-261c-4e0a-826e-162f714dd023 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:42.283292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19be6a-261c-4e0a-826e-162f714dd023 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:42.293605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1033/2055] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:01:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:01:46.671967 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7e7d1-b2b9-4b74-986e-6bf1e6101fcb None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:46.681530 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7e7d1-b2b9-4b74-986e-6bf1e6101fcb None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:46.685091 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052af39-62b9-4435-81ea-cabd7e2a8da6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:46.685726 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052af39-62b9-4435-81ea-cabd7e2a8da6 service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:46.686000 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052af39-62b9-4435-81ea-cabd7e2a8da6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:46.686311 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052af39-62b9-4435-81ea-cabd7e2a8da6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:46.704768 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b052af39-62b9-4435-81ea-cabd7e2a8da6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:46.706314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b052af39-62b9-4435-81ea-cabd7e2a8da6 service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:46.716903 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1023/2056] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:01:46 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:01:52.288580 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=36 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:52.748273 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19be6a-261c-4e0a-826e-162f714dd023 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:52.750719 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6d19be6a-261c-4e0a-826e-162f714dd023 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:52.823610 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19be6a-261c-4e0a-826e-162f714dd023 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:52.828965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d0d8d-c82a-416f-b1cf-ea148d3e3b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:52.829787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d0d8d-c82a-416f-b1cf-ea148d3e3b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:52.830153 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d0d8d-c82a-416f-b1cf-ea148d3e3b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:52.830428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d0d8d-c82a-416f-b1cf-ea148d3e3b48 admin admin] PATH_INFO: `/v3/users/812016e7e62d49c59bebe1ec24d376b2` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:52.831859 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d0d8d-c82a-416f-b1cf-ea148d3e3b48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=812016e7e62d49c59bebe1ec24d376b2)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:52.833827 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d0d8d-c82a-416f-b1cf-ea148d3e3b48 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:52.888905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-de4d0d8d-c82a-416f-b1cf-ea148d3e3b48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '812016e7e62d49c59bebe1ec24d376b2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:52.897755 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-de4d0d8d-c82a-416f-b1cf-ea148d3e3b48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '812016e7e62d49c59bebe1ec24d376b2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:52.898634 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-de4d0d8d-c82a-416f-b1cf-ea148d3e3b48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '812016e7e62d49c59bebe1ec24d376b2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:52.910809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-de4d0d8d-c82a-416f-b1cf-ea148d3e3b48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '812016e7e62d49c59bebe1ec24d376b2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:52.910809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-de4d0d8d-c82a-416f-b1cf-ea148d3e3b48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '812016e7e62d49c59bebe1ec24d376b2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:52.938768 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1034/2057] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:01:52 2022] DELETE /identity/v3/users/812016e7e62d49c59bebe1ec24d376b2 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:01:52.943540 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2e5b6-125c-4b70-9320-e3fbdf3e1816 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:52.944165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2e5b6-125c-4b70-9320-e3fbdf3e1816 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:52.944165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2e5b6-125c-4b70-9320-e3fbdf3e1816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:52.965098 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-eab2e5b6-125c-4b70-9320-e3fbdf3e1816 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:52.966159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-eab2e5b6-125c-4b70-9320-e3fbdf3e1816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:53.043151 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1024/2058] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:01:53.052924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d0d8d-c82a-416f-b1cf-ea148d3e3b48 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:53.054470 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-de4d0d8d-c82a-416f-b1cf-ea148d3e3b48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:53.152125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d0d8d-c82a-416f-b1cf-ea148d3e3b48 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:53.157013 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a61e9a-c2e4-47df-a2ae-80b10728abd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:53.157549 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a61e9a-c2e4-47df-a2ae-80b10728abd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:53.158036 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a61e9a-c2e4-47df-a2ae-80b10728abd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:53.158491 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a61e9a-c2e4-47df-a2ae-80b10728abd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:53.170366 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a61e9a-c2e4-47df-a2ae-80b10728abd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:53.172059 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a61e9a-c2e4-47df-a2ae-80b10728abd2 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:53.183108 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1035/2059] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:01:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:01:53.469219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2e5b6-125c-4b70-9320-e3fbdf3e1816 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:53.470617 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-eab2e5b6-125c-4b70-9320-e3fbdf3e1816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:53.484574 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2e5b6-125c-4b70-9320-e3fbdf3e1816 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:53.490703 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255f0ea-f373-419b-aeb3-a24704cf50b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:53.491496 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255f0ea-f373-419b-aeb3-a24704cf50b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:53.492381 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255f0ea-f373-419b-aeb3-a24704cf50b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:53.492776 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255f0ea-f373-419b-aeb3-a24704cf50b4 admin admin] PATH_INFO: `/v3/projects/34971f327bf546a8a0001c844a188141` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:53.495091 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255f0ea-f373-419b-aeb3-a24704cf50b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34971f327bf546a8a0001c844a188141)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:53.497070 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255f0ea-f373-419b-aeb3-a24704cf50b4 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:53.549657 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-1255f0ea-f373-419b-aeb3-a24704cf50b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34971f327bf546a8a0001c844a188141', 'request_id': 'req-1255f0ea-f373-419b-aeb3-a24704cf50b4'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:53.558350 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-1255f0ea-f373-419b-aeb3-a24704cf50b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34971f327bf546a8a0001c844a188141', 'request_id': 'req-1255f0ea-f373-419b-aeb3-a24704cf50b4'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:53.565477 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-1255f0ea-f373-419b-aeb3-a24704cf50b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34971f327bf546a8a0001c844a188141', 'request_id': 'req-1255f0ea-f373-419b-aeb3-a24704cf50b4'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:53.572968 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:53.578723 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-1255f0ea-f373-419b-aeb3-a24704cf50b4 admin admin] The token cache is being invalidate because project 34971f327bf546a8a0001c844a188141 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:01:53.579282 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-1255f0ea-f373-419b-aeb3-a24704cf50b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:53.580335 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-1255f0ea-f373-419b-aeb3-a24704cf50b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:53.582858 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1025/2060] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:01:53 2022] DELETE /identity/v3/projects/34971f327bf546a8a0001c844a188141 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:01:53.588270 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a61e9a-c2e4-47df-a2ae-80b10728abd2 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:53.598247 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a61e9a-c2e4-47df-a2ae-80b10728abd2 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:53.618413 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:53.661003 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce2289-1407-4682-ba36-1b442e1c5469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:53.661563 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce2289-1407-4682-ba36-1b442e1c5469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:53.661844 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce2289-1407-4682-ba36-1b442e1c5469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:53.662115 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce2289-1407-4682-ba36-1b442e1c5469 admin admin] PATH_INFO: `/v3/users/6aaa8c50971d43cbb4b6f970f5ad18d2` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:53.663273 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce2289-1407-4682-ba36-1b442e1c5469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aaa8c50971d43cbb4b6f970f5ad18d2)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:53.665298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce2289-1407-4682-ba36-1b442e1c5469 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:53.723638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-78ce2289-1407-4682-ba36-1b442e1c5469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aaa8c50971d43cbb4b6f970f5ad18d2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:53.731756 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-78ce2289-1407-4682-ba36-1b442e1c5469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aaa8c50971d43cbb4b6f970f5ad18d2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:53.732591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-78ce2289-1407-4682-ba36-1b442e1c5469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aaa8c50971d43cbb4b6f970f5ad18d2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:53.741545 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-78ce2289-1407-4682-ba36-1b442e1c5469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aaa8c50971d43cbb4b6f970f5ad18d2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:53.742186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-78ce2289-1407-4682-ba36-1b442e1c5469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aaa8c50971d43cbb4b6f970f5ad18d2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:53.761026 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1036/2061] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:01:53 2022] DELETE /identity/v3/users/6aaa8c50971d43cbb4b6f970f5ad18d2 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:01:54.077180 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255f0ea-f373-419b-aeb3-a24704cf50b4 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:54.078718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-1255f0ea-f373-419b-aeb3-a24704cf50b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:54.092190 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255f0ea-f373-419b-aeb3-a24704cf50b4 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:54.175903 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd4755-1d9a-47f1-8b46-48a38098eeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:54.176900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd4755-1d9a-47f1-8b46-48a38098eeeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:54.177376 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd4755-1d9a-47f1-8b46-48a38098eeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:54.177818 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd4755-1d9a-47f1-8b46-48a38098eeeb admin admin] PATH_INFO: `/v3/projects/b60c24c2fcaa4e599e8e925eeaf31006` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:54.179777 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd4755-1d9a-47f1-8b46-48a38098eeeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b60c24c2fcaa4e599e8e925eeaf31006)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:54.183208 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd4755-1d9a-47f1-8b46-48a38098eeeb admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:54.247442 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-4bbd4755-1d9a-47f1-8b46-48a38098eeeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b60c24c2fcaa4e599e8e925eeaf31006', 'request_id': 'req-4bbd4755-1d9a-47f1-8b46-48a38098eeeb'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:54.256529 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-4bbd4755-1d9a-47f1-8b46-48a38098eeeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b60c24c2fcaa4e599e8e925eeaf31006', 'request_id': 'req-4bbd4755-1d9a-47f1-8b46-48a38098eeeb'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:54.263821 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-4bbd4755-1d9a-47f1-8b46-48a38098eeeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b60c24c2fcaa4e599e8e925eeaf31006', 'request_id': 'req-4bbd4755-1d9a-47f1-8b46-48a38098eeeb'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:54.284681 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-4bbd4755-1d9a-47f1-8b46-48a38098eeeb admin admin] The token cache is being invalidate because project b60c24c2fcaa4e599e8e925eeaf31006 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:01:54.284987 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-4bbd4755-1d9a-47f1-8b46-48a38098eeeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:54.285671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-4bbd4755-1d9a-47f1-8b46-48a38098eeeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:54.287476 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1026/2062] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:01:54 2022] DELETE /identity/v3/projects/b60c24c2fcaa4e599e8e925eeaf31006 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:01:54.294959 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa27bd-4575-48fe-b8bb-0c9fb302534a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:54.295603 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa27bd-4575-48fe-b8bb-0c9fb302534a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:54.296033 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa27bd-4575-48fe-b8bb-0c9fb302534a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:54.320684 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-70fa27bd-4575-48fe-b8bb-0c9fb302534a None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:54.322051 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-70fa27bd-4575-48fe-b8bb-0c9fb302534a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:54.413765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1037/2063] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:01:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:01:54.425030 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffba7b4-2f35-4218-9591-4bfe80518e6c None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:54.425030 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffba7b4-2f35-4218-9591-4bfe80518e6c None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:54.425030 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffba7b4-2f35-4218-9591-4bfe80518e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:54.445647 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-9ffba7b4-2f35-4218-9591-4bfe80518e6c None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:54.447226 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-9ffba7b4-2f35-4218-9591-4bfe80518e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:54.460246 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1027/2064] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:01:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:01:54.465132 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa27bd-4575-48fe-b8bb-0c9fb302534a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:54.475591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa27bd-4575-48fe-b8bb-0c9fb302534a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:54.480348 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd560178-b81e-4aaf-a502-bcfb15ce41e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:54.480983 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd560178-b81e-4aaf-a502-bcfb15ce41e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:54.481263 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd560178-b81e-4aaf-a502-bcfb15ce41e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:54.481531 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd560178-b81e-4aaf-a502-bcfb15ce41e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:54.482555 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd560178-b81e-4aaf-a502-bcfb15ce41e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:01:54.482555 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd560178-b81e-4aaf-a502-bcfb15ce41e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:54.484399 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd560178-b81e-4aaf-a502-bcfb15ce41e5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:54.497810 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1038/2065] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:01:54.502720 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffba7b4-2f35-4218-9591-4bfe80518e6c None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:54.526403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffba7b4-2f35-4218-9591-4bfe80518e6c None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:54.540863 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cee071-c26f-4aa7-bdb1-59fb9ce752b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:54.541531 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cee071-c26f-4aa7-bdb1-59fb9ce752b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:54.541787 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cee071-c26f-4aa7-bdb1-59fb9ce752b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:54.542059 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cee071-c26f-4aa7-bdb1-59fb9ce752b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:54.542736 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cee071-c26f-4aa7-bdb1-59fb9ce752b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:54.546271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cee071-c26f-4aa7-bdb1-59fb9ce752b1 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:54.597794 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1028/2066] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:01:54 2022] POST /identity/v3/projects => generated 367 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:01:54.601948 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd560178-b81e-4aaf-a502-bcfb15ce41e5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:54.618426 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd560178-b81e-4aaf-a502-bcfb15ce41e5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:54.649826 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:54.685864 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e4ee6-c763-4ba3-ace3-81c055c33c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:54.686465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e4ee6-c763-4ba3-ace3-81c055c33c75 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:54.686731 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e4ee6-c763-4ba3-ace3-81c055c33c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:54.687003 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e4ee6-c763-4ba3-ace3-81c055c33c75 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:54.687787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e4ee6-c763-4ba3-ace3-81c055c33c75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:54.689735 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e4ee6-c763-4ba3-ace3-81c055c33c75 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:54.718948 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1039/2067] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:01:54 2022] POST /identity/v3/users => generated 284 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:01:54.720542 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cee071-c26f-4aa7-bdb1-59fb9ce752b1 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:54.731611 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cee071-c26f-4aa7-bdb1-59fb9ce752b1 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:54.736452 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0ddb1-355c-49ee-b757-e1aebf93dd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:54.737083 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0ddb1-355c-49ee-b757-e1aebf93dd32 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:54.737389 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0ddb1-355c-49ee-b757-e1aebf93dd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:54.737660 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0ddb1-355c-49ee-b757-e1aebf93dd32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:54.738383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df0ddb1-355c-49ee-b757-e1aebf93dd32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:54.740087 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df0ddb1-355c-49ee-b757-e1aebf93dd32 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:54.750810 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1029/2068] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:01:54.755101 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e4ee6-c763-4ba3-ace3-81c055c33c75 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:54.766491 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e4ee6-c763-4ba3-ace3-81c055c33c75 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:54.771911 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7326c6-6c47-4477-8513-b15655cee96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:54.772558 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7326c6-6c47-4477-8513-b15655cee96a admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:54.772842 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7326c6-6c47-4477-8513-b15655cee96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:54.773152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7326c6-6c47-4477-8513-b15655cee96a admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:54.773867 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7326c6-6c47-4477-8513-b15655cee96a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:54.775481 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7326c6-6c47-4477-8513-b15655cee96a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:54.786660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1040/2069] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:01:54.790977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0ddb1-355c-49ee-b757-e1aebf93dd32 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:54.801526 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0ddb1-355c-49ee-b757-e1aebf93dd32 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:54.807751 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ee01d-9c58-4da5-8cf1-9453cbcd2d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:54.808573 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ee01d-9c58-4da5-8cf1-9453cbcd2d51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:54.808883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ee01d-9c58-4da5-8cf1-9453cbcd2d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:54.809222 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ee01d-9c58-4da5-8cf1-9453cbcd2d51 admin admin] PATH_INFO: `/v3/projects/523d4e75364b40b1866a2973ce2ddc63/users/120db8b17ba94ff6b2edd08ea16bf739/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:54.830545 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ee01d-9c58-4da5-8cf1-9453cbcd2d51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=523d4e75364b40b1866a2973ce2ddc63, user_id=120db8b17ba94ff6b2edd08ea16bf739, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:54.833390 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ee01d-9c58-4da5-8cf1-9453cbcd2d51 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:54.852917 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1030/2070] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:01:54 2022] PUT /identity/v3/projects/523d4e75364b40b1866a2973ce2ddc63/users/120db8b17ba94ff6b2edd08ea16bf739/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:01:54.864138 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a38e8-622f-42a1-b946-21212a3c07b9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:54.864791 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a38e8-622f-42a1-b946-21212a3c07b9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:54.865132 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a38e8-622f-42a1-b946-21212a3c07b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:54.894316 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d59a38e8-622f-42a1-b946-21212a3c07b9 None None] MFA Rules not processed for user `120db8b17ba94ff6b2edd08ea16bf739`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:54.895760 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d59a38e8-622f-42a1-b946-21212a3c07b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:54.968393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1041/2071] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:01:54 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:01:54.973137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ee01d-9c58-4da5-8cf1-9453cbcd2d51 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:55.013092 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:55.074508 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ee01d-9c58-4da5-8cf1-9453cbcd2d51 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:55.079351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ec716-4f21-4826-90dd-2439016f862c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:55.080048 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ec716-4f21-4826-90dd-2439016f862c admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:55.080293 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ec716-4f21-4826-90dd-2439016f862c admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:55.080719 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ec716-4f21-4826-90dd-2439016f862c admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:55.081417 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ec716-4f21-4826-90dd-2439016f862c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:55.083331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ec716-4f21-4826-90dd-2439016f862c admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:55.111666 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1031/2072] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:01:54 2022] POST /identity/v3/projects => generated 369 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:01:55.115846 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a38e8-622f-42a1-b946-21212a3c07b9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:55.127888 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a38e8-622f-42a1-b946-21212a3c07b9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:55.186238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d42827d-d639-47ac-868c-b5ddce15f120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:55.186878 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42827d-d639-47ac-868c-b5ddce15f120 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:55.187162 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42827d-d639-47ac-868c-b5ddce15f120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:55.187463 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42827d-d639-47ac-868c-b5ddce15f120 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:55.188166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d42827d-d639-47ac-868c-b5ddce15f120 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:55.190098 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d42827d-d639-47ac-868c-b5ddce15f120 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:55.216150 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1042/2073] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:01:55 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:01:55.217142 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ec716-4f21-4826-90dd-2439016f862c admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:55.229583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ec716-4f21-4826-90dd-2439016f862c admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:55.236887 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87254134-4b61-4007-a00d-3738389a4cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:55.237637 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87254134-4b61-4007-a00d-3738389a4cac admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:55.238007 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87254134-4b61-4007-a00d-3738389a4cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:55.238286 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87254134-4b61-4007-a00d-3738389a4cac admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:55.239009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87254134-4b61-4007-a00d-3738389a4cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:55.240738 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87254134-4b61-4007-a00d-3738389a4cac admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:55.253227 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1032/2074] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:01:55.257916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d42827d-d639-47ac-868c-b5ddce15f120 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:55.269264 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d42827d-d639-47ac-868c-b5ddce15f120 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:55.273910 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482e957-f0fc-4d00-b583-751788475f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:55.274718 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482e957-f0fc-4d00-b583-751788475f0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:55.275002 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482e957-f0fc-4d00-b583-751788475f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:55.275285 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482e957-f0fc-4d00-b583-751788475f0b admin admin] PATH_INFO: `/v3/projects/1e080f38798142a28c6afa3aa528ef9f/users/0180ae87580a4356b6d59e01c3639388/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:55.293379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482e957-f0fc-4d00-b583-751788475f0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e080f38798142a28c6afa3aa528ef9f, user_id=0180ae87580a4356b6d59e01c3639388, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:55.296277 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482e957-f0fc-4d00-b583-751788475f0b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:55.316903 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1043/2075] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:01:55 2022] PUT /identity/v3/projects/1e080f38798142a28c6afa3aa528ef9f/users/0180ae87580a4356b6d59e01c3639388/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:01:55.321206 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87254134-4b61-4007-a00d-3738389a4cac admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:55.332463 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87254134-4b61-4007-a00d-3738389a4cac admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:55.408701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06c7ff-691a-41fe-a2a9-b8a0466b166d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:55.409349 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06c7ff-691a-41fe-a2a9-b8a0466b166d admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:55.409633 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06c7ff-691a-41fe-a2a9-b8a0466b166d admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:55.409906 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06c7ff-691a-41fe-a2a9-b8a0466b166d admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:55.410668 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be06c7ff-691a-41fe-a2a9-b8a0466b166d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:55.412382 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be06c7ff-691a-41fe-a2a9-b8a0466b166d admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:55.422967 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1033/2076] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:01:55.427258 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482e957-f0fc-4d00-b583-751788475f0b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:55.433295 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:55.444235 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482e957-f0fc-4d00-b583-751788475f0b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:55.449655 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05821ed7-3862-407b-b05d-da18deb07d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:55.450349 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05821ed7-3862-407b-b05d-da18deb07d87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:55.450641 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05821ed7-3862-407b-b05d-da18deb07d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:55.450909 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05821ed7-3862-407b-b05d-da18deb07d87 admin admin] PATH_INFO: `/v3/projects/1e080f38798142a28c6afa3aa528ef9f/users/0180ae87580a4356b6d59e01c3639388/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:55.453138 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05821ed7-3862-407b-b05d-da18deb07d87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e080f38798142a28c6afa3aa528ef9f, user_id=0180ae87580a4356b6d59e01c3639388, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:55.455825 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05821ed7-3862-407b-b05d-da18deb07d87 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:55.468324 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1044/2077] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:01:55 2022] PUT /identity/v3/projects/1e080f38798142a28c6afa3aa528ef9f/users/0180ae87580a4356b6d59e01c3639388/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:01:55.480213 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94764c8c-3bf8-4867-b1ed-1c81de5f4cc5 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:55.480507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94764c8c-3bf8-4867-b1ed-1c81de5f4cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:55.480785 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94764c8c-3bf8-4867-b1ed-1c81de5f4cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:55.512690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-94764c8c-3bf8-4867-b1ed-1c81de5f4cc5 None None] MFA Rules not processed for user `0180ae87580a4356b6d59e01c3639388`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:55.514577 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-94764c8c-3bf8-4867-b1ed-1c81de5f4cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:55.599356 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1034/2078] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:01:55 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:01:55.625010 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05821ed7-3862-407b-b05d-da18deb07d87 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:55.627272 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-05821ed7-3862-407b-b05d-da18deb07d87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:55.712714 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05821ed7-3862-407b-b05d-da18deb07d87 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:55.716518 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d34fce-e9d7-4923-96c2-e0aa3f555ae7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:55.717340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d34fce-e9d7-4923-96c2-e0aa3f555ae7 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:55.717636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d34fce-e9d7-4923-96c2-e0aa3f555ae7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:55.718101 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d34fce-e9d7-4923-96c2-e0aa3f555ae7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:55.728568 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d34fce-e9d7-4923-96c2-e0aa3f555ae7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:55.729978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d34fce-e9d7-4923-96c2-e0aa3f555ae7 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:55.740630 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1045/2079] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:01:55 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:01:55.793179 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94764c8c-3bf8-4867-b1ed-1c81de5f4cc5 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:55.794841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-94764c8c-3bf8-4867-b1ed-1c81de5f4cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:55.807975 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94764c8c-3bf8-4867-b1ed-1c81de5f4cc5 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:55.812915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf247b-dcdf-4c45-9529-c979c7d8633a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:55.814315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf247b-dcdf-4c45-9529-c979c7d8633a service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:55.814609 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf247b-dcdf-4c45-9529-c979c7d8633a service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:55.814890 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf247b-dcdf-4c45-9529-c979c7d8633a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:55.838532 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:55.886129 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf247b-dcdf-4c45-9529-c979c7d8633a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:55.887670 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf247b-dcdf-4c45-9529-c979c7d8633a service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:55.898159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1035/2080] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:01:55 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:01:58.508408 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001b743-6494-4a07-ba4c-fe46656dbbb1 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:58.508931 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001b743-6494-4a07-ba4c-fe46656dbbb1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:58.508931 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001b743-6494-4a07-ba4c-fe46656dbbb1 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:58.510265 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1046/2081] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jul 26 05:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 05:01:58.537698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd6e50-eba4-4c02-8069-c98b6abbce03 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:58.538342 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd6e50-eba4-4c02-8069-c98b6abbce03 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:58.538483 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd6e50-eba4-4c02-8069-c98b6abbce03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:58.561313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-6fdd6e50-eba4-4c02-8069-c98b6abbce03 None None] MFA Rules not processed for user `5049dba7ffde47b8af3e6fd55b5703e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:58.562582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-6fdd6e50-eba4-4c02-8069-c98b6abbce03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:58.579251 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001b743-6494-4a07-ba4c-fe46656dbbb1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:58.581388 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5001b743-6494-4a07-ba4c-fe46656dbbb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:58.633648 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1036/2082] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jul 26 05:01:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:01:58.647184 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd6e50-eba4-4c02-8069-c98b6abbce03 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:58.648840 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-6fdd6e50-eba4-4c02-8069-c98b6abbce03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:58.662870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd6e50-eba4-4c02-8069-c98b6abbce03 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:58.669484 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff5faa-8a4a-4ff5-9ba3-112512d20539 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5049dba7ffde47b8af3e6fd55b5703e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:58.670349 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff5faa-8a4a-4ff5-9ba3-112512d20539 service placement] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:58.670528 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff5faa-8a4a-4ff5-9ba3-112512d20539 service placement] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:58.670796 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff5faa-8a4a-4ff5-9ba3-112512d20539 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:58.673164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001b743-6494-4a07-ba4c-fe46656dbbb1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:58.680489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff5faa-8a4a-4ff5-9ba3-112512d20539 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:58.682085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff5faa-8a4a-4ff5-9ba3-112512d20539 service placement] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:58.684095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471c4f9-1fed-4829-a912-b30926620daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:58.684906 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471c4f9-1fed-4829-a912-b30926620daa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:58.685298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471c4f9-1fed-4829-a912-b30926620daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:58.685680 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471c4f9-1fed-4829-a912-b30926620daa admin admin] PATH_INFO: `/v3/users/f4e86ac513aa4feda193e134c0bc9e79` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:58.687124 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d471c4f9-1fed-4829-a912-b30926620daa admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4e86ac513aa4feda193e134c0bc9e79)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:58.689400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d471c4f9-1fed-4829-a912-b30926620daa admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:58.696318 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1037/2083] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:01:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:01:58.748794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d471c4f9-1fed-4829-a912-b30926620daa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4e86ac513aa4feda193e134c0bc9e79'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:58.759578 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d471c4f9-1fed-4829-a912-b30926620daa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4e86ac513aa4feda193e134c0bc9e79'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:58.760930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d471c4f9-1fed-4829-a912-b30926620daa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4e86ac513aa4feda193e134c0bc9e79'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:58.774456 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d471c4f9-1fed-4829-a912-b30926620daa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4e86ac513aa4feda193e134c0bc9e79'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:58.775198 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d471c4f9-1fed-4829-a912-b30926620daa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4e86ac513aa4feda193e134c0bc9e79'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:58.794715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1047/2084] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:01:58 2022] DELETE /identity/v3/users/f4e86ac513aa4feda193e134c0bc9e79 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:01:59.038973 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff5faa-8a4a-4ff5-9ba3-112512d20539 service placement] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:59.055896 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff5faa-8a4a-4ff5-9ba3-112512d20539 service placement] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:59.165960 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1763c42-abcf-43b9-ad0d-a9cab5b0f7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:59.166727 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1763c42-abcf-43b9-ad0d-a9cab5b0f7fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:59.167012 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1763c42-abcf-43b9-ad0d-a9cab5b0f7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:59.167317 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1763c42-abcf-43b9-ad0d-a9cab5b0f7fa admin admin] PATH_INFO: `/v3/projects/981b76b8fd394154a07921073ab08ed6` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:59.168581 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1763c42-abcf-43b9-ad0d-a9cab5b0f7fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=981b76b8fd394154a07921073ab08ed6)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:59.174468 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1763c42-abcf-43b9-ad0d-a9cab5b0f7fa admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:59.231439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:59.239448 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-b1763c42-abcf-43b9-ad0d-a9cab5b0f7fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '981b76b8fd394154a07921073ab08ed6', 'request_id': 'req-b1763c42-abcf-43b9-ad0d-a9cab5b0f7fa'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:59.250416 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-b1763c42-abcf-43b9-ad0d-a9cab5b0f7fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '981b76b8fd394154a07921073ab08ed6', 'request_id': 'req-b1763c42-abcf-43b9-ad0d-a9cab5b0f7fa'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:59.257613 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-b1763c42-abcf-43b9-ad0d-a9cab5b0f7fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '981b76b8fd394154a07921073ab08ed6', 'request_id': 'req-b1763c42-abcf-43b9-ad0d-a9cab5b0f7fa'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:59.273848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-b1763c42-abcf-43b9-ad0d-a9cab5b0f7fa admin admin] The token cache is being invalidate because project 981b76b8fd394154a07921073ab08ed6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:01:59.274195 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-b1763c42-abcf-43b9-ad0d-a9cab5b0f7fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:59.274955 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-b1763c42-abcf-43b9-ad0d-a9cab5b0f7fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:59.276789 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1038/2085] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:01:59 2022] DELETE /identity/v3/projects/981b76b8fd394154a07921073ab08ed6 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:01:59.282115 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3a9c1-9c81-44a1-af05-287abc2550a7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:59.282405 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3a9c1-9c81-44a1-af05-287abc2550a7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:59.282698 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3a9c1-9c81-44a1-af05-287abc2550a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:59.315639 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-67a3a9c1-9c81-44a1-af05-287abc2550a7 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:59.317604 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-67a3a9c1-9c81-44a1-af05-287abc2550a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:59.325261 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:59.332596 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1763c42-abcf-43b9-ad0d-a9cab5b0f7fa admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:59.332596 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-b1763c42-abcf-43b9-ad0d-a9cab5b0f7fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:59.401870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1048/2086] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:01:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:01:59.409095 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce796c4-9fbd-443e-affe-3f657f976b65 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:59.409396 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce796c4-9fbd-443e-affe-3f657f976b65 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:59.409672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce796c4-9fbd-443e-affe-3f657f976b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:59.426497 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1763c42-abcf-43b9-ad0d-a9cab5b0f7fa admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:59.430552 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-cce796c4-9fbd-443e-affe-3f657f976b65 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:59.431908 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cce796c4-9fbd-443e-affe-3f657f976b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:59.435694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015dec6-d362-41e6-909c-f4da751035d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:59.436977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015dec6-d362-41e6-909c-f4da751035d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:59.437359 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015dec6-d362-41e6-909c-f4da751035d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:59.437831 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015dec6-d362-41e6-909c-f4da751035d5 admin admin] PATH_INFO: `/v3/users/7ca6673bd7ae41edade8656e34330d12` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:59.439792 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8015dec6-d362-41e6-909c-f4da751035d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ca6673bd7ae41edade8656e34330d12)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:59.443889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1049/2087] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:01:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:01:59.444037 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8015dec6-d362-41e6-909c-f4da751035d5 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:59.445049 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce796c4-9fbd-443e-affe-3f657f976b65 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:59.458115 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce796c4-9fbd-443e-affe-3f657f976b65 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:59.465801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d227f4-9f64-479f-9779-d25cb8e89d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:59.466415 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d227f4-9f64-479f-9779-d25cb8e89d68 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:59.466726 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d227f4-9f64-479f-9779-d25cb8e89d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:59.466965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d227f4-9f64-479f-9779-d25cb8e89d68 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:59.468562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d227f4-9f64-479f-9779-d25cb8e89d68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:01:59.471816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d227f4-9f64-479f-9779-d25cb8e89d68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:59.471816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d227f4-9f64-479f-9779-d25cb8e89d68 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:59.499827 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1050/2088] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:01:59.501507 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d227f4-9f64-479f-9779-d25cb8e89d68 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:59.513753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8015dec6-d362-41e6-909c-f4da751035d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ca6673bd7ae41edade8656e34330d12'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:59.514096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d227f4-9f64-479f-9779-d25cb8e89d68 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:59.518766 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d687d-74de-40fe-87d2-f159ed1e2d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:59.519400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d687d-74de-40fe-87d2-f159ed1e2d59 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:59.519717 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d687d-74de-40fe-87d2-f159ed1e2d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:59.520071 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d687d-74de-40fe-87d2-f159ed1e2d59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:59.520662 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d687d-74de-40fe-87d2-f159ed1e2d59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:59.522585 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d687d-74de-40fe-87d2-f159ed1e2d59 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:59.529688 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8015dec6-d362-41e6-909c-f4da751035d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ca6673bd7ae41edade8656e34330d12'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:59.543919 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8015dec6-d362-41e6-909c-f4da751035d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ca6673bd7ae41edade8656e34330d12'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:59.545132 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8015dec6-d362-41e6-909c-f4da751035d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ca6673bd7ae41edade8656e34330d12'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:59.545921 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8015dec6-d362-41e6-909c-f4da751035d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ca6673bd7ae41edade8656e34330d12'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:59.557601 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1051/2089] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:01:59 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:01:59.562586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d687d-74de-40fe-87d2-f159ed1e2d59 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:59.564882 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b13d687d-74de-40fe-87d2-f159ed1e2d59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:59.575216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1039/2090] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:01:59 2022] DELETE /identity/v3/users/7ca6673bd7ae41edade8656e34330d12 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:01:59.580309 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd737ec2-ded5-42c6-9085-2b4b39f0e5bf None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:59.580559 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd737ec2-ded5-42c6-9085-2b4b39f0e5bf None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:59.580829 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd737ec2-ded5-42c6-9085-2b4b39f0e5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:59.604813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-fd737ec2-ded5-42c6-9085-2b4b39f0e5bf None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:59.606220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-fd737ec2-ded5-42c6-9085-2b4b39f0e5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:59.637512 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d687d-74de-40fe-87d2-f159ed1e2d59 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:59.653633 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:59.673224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1040/2091] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:01:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:01:59.677609 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd737ec2-ded5-42c6-9085-2b4b39f0e5bf None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:59.679028 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-fd737ec2-ded5-42c6-9085-2b4b39f0e5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:59.694688 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd737ec2-ded5-42c6-9085-2b4b39f0e5bf None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:59.699425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4879241-f0eb-45ea-81d2-6ad5df41e1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:59.699809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee9253-5925-4560-bd10-d0ddbc50ae20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:59.699975 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4879241-f0eb-45ea-81d2-6ad5df41e1ba admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:59.700254 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4879241-f0eb-45ea-81d2-6ad5df41e1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:59.700462 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee9253-5925-4560-bd10-d0ddbc50ae20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:59.700592 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4879241-f0eb-45ea-81d2-6ad5df41e1ba admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:59.700921 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee9253-5925-4560-bd10-d0ddbc50ae20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:59.701102 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee9253-5925-4560-bd10-d0ddbc50ae20 admin admin] PATH_INFO: `/v3/users/120db8b17ba94ff6b2edd08ea16bf739` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:59.701191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4879241-f0eb-45ea-81d2-6ad5df41e1ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:59.702387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee9253-5925-4560-bd10-d0ddbc50ae20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=120db8b17ba94ff6b2edd08ea16bf739)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:59.703094 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4879241-f0eb-45ea-81d2-6ad5df41e1ba admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:59.704382 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee9253-5925-4560-bd10-d0ddbc50ae20 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:59.725717 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1052/2092] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:01:59 2022] POST /identity/v3/users => generated 291 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:01:59.726822 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4879241-f0eb-45ea-81d2-6ad5df41e1ba admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:59.728998 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d4879241-f0eb-45ea-81d2-6ad5df41e1ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:59.743671 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:01:59.762080 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f8ee9253-5925-4560-bd10-d0ddbc50ae20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '120db8b17ba94ff6b2edd08ea16bf739'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:59.769600 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f8ee9253-5925-4560-bd10-d0ddbc50ae20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '120db8b17ba94ff6b2edd08ea16bf739'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:59.778742 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f8ee9253-5925-4560-bd10-d0ddbc50ae20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '120db8b17ba94ff6b2edd08ea16bf739'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:59.779443 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f8ee9253-5925-4560-bd10-d0ddbc50ae20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '120db8b17ba94ff6b2edd08ea16bf739'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:59.780215 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f8ee9253-5925-4560-bd10-d0ddbc50ae20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '120db8b17ba94ff6b2edd08ea16bf739'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:01:59.796974 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4879241-f0eb-45ea-81d2-6ad5df41e1ba admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:59.801263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1041/2093] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:01:59 2022] DELETE /identity/v3/users/120db8b17ba94ff6b2edd08ea16bf739 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:01:59.802194 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee9253-5925-4560-bd10-d0ddbc50ae20 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:59.803533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-f8ee9253-5925-4560-bd10-d0ddbc50ae20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:59.859170 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141e738-e6db-42b4-8e1a-7397e9f3ffc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:59.860064 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141e738-e6db-42b4-8e1a-7397e9f3ffc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:59.860359 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141e738-e6db-42b4-8e1a-7397e9f3ffc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:59.860637 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141e738-e6db-42b4-8e1a-7397e9f3ffc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:59.862533 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5141e738-e6db-42b4-8e1a-7397e9f3ffc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:59.894042 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee9253-5925-4560-bd10-d0ddbc50ae20 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:59.904444 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681053d-5bbc-41a3-9e65-6826719c896b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:59.905668 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681053d-5bbc-41a3-9e65-6826719c896b admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:59.906031 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681053d-5bbc-41a3-9e65-6826719c896b admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:59.906395 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681053d-5bbc-41a3-9e65-6826719c896b admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:59.907216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f681053d-5bbc-41a3-9e65-6826719c896b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:59.910936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f681053d-5bbc-41a3-9e65-6826719c896b admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:59.924283 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1042/2094] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:01:59.925545 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd8a33-5d67-4996-b496-930bd48eb90b None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:59.925908 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd8a33-5d67-4996-b496-930bd48eb90b None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:59.926254 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd8a33-5d67-4996-b496-930bd48eb90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:59.950639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-edbd8a33-5d67-4996-b496-930bd48eb90b None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:01:59.952869 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-edbd8a33-5d67-4996-b496-930bd48eb90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:01:59.963677 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1043/2095] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:01:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:01:59.963677 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd8a33-5d67-4996-b496-930bd48eb90b None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:01:59.975577 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd8a33-5d67-4996-b496-930bd48eb90b None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:01:59.984334 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cb61b-42dd-4564-9f78-e46bbe24aa3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:01:59.985154 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cb61b-42dd-4564-9f78-e46bbe24aa3e admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:01:59.985440 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cb61b-42dd-4564-9f78-e46bbe24aa3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:01:59.985858 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cb61b-42dd-4564-9f78-e46bbe24aa3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:01:59.986578 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997cb61b-42dd-4564-9f78-e46bbe24aa3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:59.989246 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997cb61b-42dd-4564-9f78-e46bbe24aa3e admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:01:59.989984 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5141e738-e6db-42b4-8e1a-7397e9f3ffc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:01:59.991672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5141e738-e6db-42b4-8e1a-7397e9f3ffc6 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:00.001612 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1044/2096] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:00.002390 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1053/2097] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:01:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:02:00.006006 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cb61b-42dd-4564-9f78-e46bbe24aa3e admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:00.006986 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141e738-e6db-42b4-8e1a-7397e9f3ffc6 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:00.008632 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5141e738-e6db-42b4-8e1a-7397e9f3ffc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:00.020899 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141e738-e6db-42b4-8e1a-7397e9f3ffc6 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:00.026552 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f2f95-d9ed-41ca-8cf3-d851407f969e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:00.029659 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f2f95-d9ed-41ca-8cf3-d851407f969e service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:00.029659 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f2f95-d9ed-41ca-8cf3-d851407f969e service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:00.029659 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f2f95-d9ed-41ca-8cf3-d851407f969e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:00.030979 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cb61b-42dd-4564-9f78-e46bbe24aa3e admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:00.041037 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f2f95-d9ed-41ca-8cf3-d851407f969e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:00.041705 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c39193-285f-4550-bb33-e11591e5203c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:00.042765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f2f95-d9ed-41ca-8cf3-d851407f969e service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:00.043356 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c39193-285f-4550-bb33-e11591e5203c admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:00.043822 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c39193-285f-4550-bb33-e11591e5203c admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:00.044943 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c39193-285f-4550-bb33-e11591e5203c admin admin] PATH_INFO: `/v3/projects/bd2ad6590d394a99850423f57311fa26/users/08a8b9fcf94a4be39ec5468bc24c6dfd/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:00.058614 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1054/2098] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:02:00.073872 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c39193-285f-4550-bb33-e11591e5203c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd2ad6590d394a99850423f57311fa26, user_id=08a8b9fcf94a4be39ec5468bc24c6dfd, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:00.077291 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c39193-285f-4550-bb33-e11591e5203c admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:00.100678 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1045/2099] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:00 2022] PUT /identity/v3/projects/bd2ad6590d394a99850423f57311fa26/users/08a8b9fcf94a4be39ec5468bc24c6dfd/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:00.111658 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e36165-1ed5-4dab-89e5-9e72cbbed0e9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:00.112173 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e36165-1ed5-4dab-89e5-9e72cbbed0e9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:00.112173 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e36165-1ed5-4dab-89e5-9e72cbbed0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:00.185504 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f3e36165-1ed5-4dab-89e5-9e72cbbed0e9 None None] MFA Rules not processed for user `08a8b9fcf94a4be39ec5468bc24c6dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:00.187482 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f3e36165-1ed5-4dab-89e5-9e72cbbed0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:00.257298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:00.292713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1055/2100] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:00.297173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c39193-285f-4550-bb33-e11591e5203c admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:00.327698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:00.438558 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c39193-285f-4550-bb33-e11591e5203c admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:00.443990 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d9967-e6d5-45be-834f-4b1bcbedba57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:00.444671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d9967-e6d5-45be-834f-4b1bcbedba57 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:00.445003 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d9967-e6d5-45be-834f-4b1bcbedba57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:00.445239 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d9967-e6d5-45be-834f-4b1bcbedba57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:00.445921 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d9967-e6d5-45be-834f-4b1bcbedba57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:00.447909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d9967-e6d5-45be-834f-4b1bcbedba57 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:00.475469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e36165-1ed5-4dab-89e5-9e72cbbed0e9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:00.477510 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f3e36165-1ed5-4dab-89e5-9e72cbbed0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:00.497652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1046/2101] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:02:00 2022] POST /identity/v3/projects => generated 379 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:00.498992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d9967-e6d5-45be-834f-4b1bcbedba57 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:00.505819 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e36165-1ed5-4dab-89e5-9e72cbbed0e9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:00.573989 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d9967-e6d5-45be-834f-4b1bcbedba57 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:00.577665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5234a9-ff88-42e1-b23e-cd07fbd725b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:00.578348 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5234a9-ff88-42e1-b23e-cd07fbd725b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:00.578702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5234a9-ff88-42e1-b23e-cd07fbd725b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:00.578853 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778b872-df81-4d1f-8163-231b0409087b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:00.579129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5234a9-ff88-42e1-b23e-cd07fbd725b4 admin admin] PATH_INFO: `/v3/projects/523d4e75364b40b1866a2973ce2ddc63` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:00.579821 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778b872-df81-4d1f-8163-231b0409087b admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:00.580108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778b872-df81-4d1f-8163-231b0409087b admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:00.580413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778b872-df81-4d1f-8163-231b0409087b admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:00.580519 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5234a9-ff88-42e1-b23e-cd07fbd725b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=523d4e75364b40b1866a2973ce2ddc63)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:00.581084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778b872-df81-4d1f-8163-231b0409087b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:00.582606 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5234a9-ff88-42e1-b23e-cd07fbd725b4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:00.583020 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778b872-df81-4d1f-8163-231b0409087b admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:00.614951 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1047/2102] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:02:00 2022] POST /identity/v3/users => generated 296 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:00.616308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778b872-df81-4d1f-8163-231b0409087b admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:00.634372 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778b872-df81-4d1f-8163-231b0409087b admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:00.653023 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fe5234a9-ff88-42e1-b23e-cd07fbd725b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '523d4e75364b40b1866a2973ce2ddc63', 'request_id': 'req-fe5234a9-ff88-42e1-b23e-cd07fbd725b4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:00.660689 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fe5234a9-ff88-42e1-b23e-cd07fbd725b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '523d4e75364b40b1866a2973ce2ddc63', 'request_id': 'req-fe5234a9-ff88-42e1-b23e-cd07fbd725b4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:00.668975 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fe5234a9-ff88-42e1-b23e-cd07fbd725b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '523d4e75364b40b1866a2973ce2ddc63', 'request_id': 'req-fe5234a9-ff88-42e1-b23e-cd07fbd725b4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:00.687111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fe5234a9-ff88-42e1-b23e-cd07fbd725b4 admin admin] The token cache is being invalidate because project 523d4e75364b40b1866a2973ce2ddc63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:00.690340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fe5234a9-ff88-42e1-b23e-cd07fbd725b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:00.691030 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fe5234a9-ff88-42e1-b23e-cd07fbd725b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:00.696017 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1056/2103] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:00 2022] DELETE /identity/v3/projects/523d4e75364b40b1866a2973ce2ddc63 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:00.697203 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5234a9-ff88-42e1-b23e-cd07fbd725b4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:00.699220 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fe5234a9-ff88-42e1-b23e-cd07fbd725b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:00.725978 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b323e-b0e5-4833-ad68-fc669138272d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:00.726773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b323e-b0e5-4833-ad68-fc669138272d admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:00.727136 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b323e-b0e5-4833-ad68-fc669138272d admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:00.727414 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b323e-b0e5-4833-ad68-fc669138272d admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:00.728259 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039b323e-b0e5-4833-ad68-fc669138272d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:00.729793 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039b323e-b0e5-4833-ad68-fc669138272d admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:00.742150 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1048/2104] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:00.743249 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b323e-b0e5-4833-ad68-fc669138272d admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:00.744973 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-039b323e-b0e5-4833-ad68-fc669138272d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:00.759470 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b323e-b0e5-4833-ad68-fc669138272d admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:00.764617 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb97aa9-d41a-49fd-a3bb-3788aabb4fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:00.765280 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb97aa9-d41a-49fd-a3bb-3788aabb4fe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:00.765560 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb97aa9-d41a-49fd-a3bb-3788aabb4fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:00.765836 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb97aa9-d41a-49fd-a3bb-3788aabb4fe8 admin admin] PATH_INFO: `/v3/users/0180ae87580a4356b6d59e01c3639388` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:00.767029 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb97aa9-d41a-49fd-a3bb-3788aabb4fe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0180ae87580a4356b6d59e01c3639388)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:00.769075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb97aa9-d41a-49fd-a3bb-3788aabb4fe8 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:00.794195 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:00.808661 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5234a9-ff88-42e1-b23e-cd07fbd725b4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:00.813691 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08fff8-1526-42b2-a92d-9817352a7c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:00.814354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08fff8-1526-42b2-a92d-9817352a7c5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:00.814795 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08fff8-1526-42b2-a92d-9817352a7c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:00.815081 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08fff8-1526-42b2-a92d-9817352a7c5b admin admin] PATH_INFO: `/v3/projects/1ae12dd973424019b41445b6f0842f32` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:00.816299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd08fff8-1526-42b2-a92d-9817352a7c5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ae12dd973424019b41445b6f0842f32)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:00.818255 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd08fff8-1526-42b2-a92d-9817352a7c5b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:00.824410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9eb97aa9-d41a-49fd-a3bb-3788aabb4fe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0180ae87580a4356b6d59e01c3639388'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:00.832400 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9eb97aa9-d41a-49fd-a3bb-3788aabb4fe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0180ae87580a4356b6d59e01c3639388'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:00.842271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9eb97aa9-d41a-49fd-a3bb-3788aabb4fe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0180ae87580a4356b6d59e01c3639388'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:00.843300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9eb97aa9-d41a-49fd-a3bb-3788aabb4fe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0180ae87580a4356b6d59e01c3639388'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:00.844198 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:00.844923 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9eb97aa9-d41a-49fd-a3bb-3788aabb4fe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0180ae87580a4356b6d59e01c3639388'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:00.866094 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1049/2105] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:02:00 2022] DELETE /identity/v3/users/0180ae87580a4356b6d59e01c3639388 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:00.867118 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb97aa9-d41a-49fd-a3bb-3788aabb4fe8 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:00.868558 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-9eb97aa9-d41a-49fd-a3bb-3788aabb4fe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:00.872395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fd08fff8-1526-42b2-a92d-9817352a7c5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ae12dd973424019b41445b6f0842f32', 'request_id': 'req-fd08fff8-1526-42b2-a92d-9817352a7c5b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:00.880753 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fd08fff8-1526-42b2-a92d-9817352a7c5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ae12dd973424019b41445b6f0842f32', 'request_id': 'req-fd08fff8-1526-42b2-a92d-9817352a7c5b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:00.890097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fd08fff8-1526-42b2-a92d-9817352a7c5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ae12dd973424019b41445b6f0842f32', 'request_id': 'req-fd08fff8-1526-42b2-a92d-9817352a7c5b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:00.902706 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fd08fff8-1526-42b2-a92d-9817352a7c5b admin admin] The token cache is being invalidate because project 1ae12dd973424019b41445b6f0842f32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:00.903326 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fd08fff8-1526-42b2-a92d-9817352a7c5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:00.903794 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fd08fff8-1526-42b2-a92d-9817352a7c5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:00.905729 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1057/2106] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:00 2022] DELETE /identity/v3/projects/1ae12dd973424019b41445b6f0842f32 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:00.909756 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08fff8-1526-42b2-a92d-9817352a7c5b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:00.911222 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fd08fff8-1526-42b2-a92d-9817352a7c5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:00.950935 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb97aa9-d41a-49fd-a3bb-3788aabb4fe8 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:00.958227 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd1d2b-56aa-4b26-ae8b-3aec536a8657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:00.959697 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd1d2b-56aa-4b26-ae8b-3aec536a8657 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:00.960344 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd1d2b-56aa-4b26-ae8b-3aec536a8657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:00.960840 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd1d2b-56aa-4b26-ae8b-3aec536a8657 admin admin] PATH_INFO: `/v3/projects/1d81a060457a4778806d9a4729384bf1/users/baf46b1f78bf46b5ad632b347d4fb8ea/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:00.983583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd1d2b-56aa-4b26-ae8b-3aec536a8657 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d81a060457a4778806d9a4729384bf1, user_id=baf46b1f78bf46b5ad632b347d4fb8ea, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:00.987884 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd1d2b-56aa-4b26-ae8b-3aec536a8657 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:01.000455 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08fff8-1526-42b2-a92d-9817352a7c5b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:01.008237 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1050/2107] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:00 2022] PUT /identity/v3/projects/1d81a060457a4778806d9a4729384bf1/users/baf46b1f78bf46b5ad632b347d4fb8ea/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:01.009292 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd1d2b-56aa-4b26-ae8b-3aec536a8657 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:01.011344 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-65bd1d2b-56aa-4b26-ae8b-3aec536a8657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:01.076888 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0baa5d8-aca3-4272-b1a4-58252bc03f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:01.077489 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0baa5d8-aca3-4272-b1a4-58252bc03f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:01.077771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0baa5d8-aca3-4272-b1a4-58252bc03f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:01.078041 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0baa5d8-aca3-4272-b1a4-58252bc03f3e admin admin] PATH_INFO: `/v3/users/73f8d4f76dc246388576b6c863116b24` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:01.079487 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0baa5d8-aca3-4272-b1a4-58252bc03f3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=73f8d4f76dc246388576b6c863116b24)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:01.082035 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0baa5d8-aca3-4272-b1a4-58252bc03f3e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:01.096166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd1d2b-56aa-4b26-ae8b-3aec536a8657 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:01.103494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebdd27e-6180-4e79-963c-9b448df11abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:01.104166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdd27e-6180-4e79-963c-9b448df11abc admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:01.104452 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdd27e-6180-4e79-963c-9b448df11abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:01.104728 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdd27e-6180-4e79-963c-9b448df11abc admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:01.106254 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebdd27e-6180-4e79-963c-9b448df11abc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:01.107956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebdd27e-6180-4e79-963c-9b448df11abc admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:01.121908 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1051/2108] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:01.126339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebdd27e-6180-4e79-963c-9b448df11abc admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:01.138613 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebdd27e-6180-4e79-963c-9b448df11abc admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:01.143527 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7f4135-b3e6-4cf6-a12f-bdf090e2ad09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:01.144612 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f4135-b3e6-4cf6-a12f-bdf090e2ad09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:01.144612 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f4135-b3e6-4cf6-a12f-bdf090e2ad09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:01.144765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f4135-b3e6-4cf6-a12f-bdf090e2ad09 admin admin] PATH_INFO: `/v3/projects/1d81a060457a4778806d9a4729384bf1/users/baf46b1f78bf46b5ad632b347d4fb8ea/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:01.146776 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7f4135-b3e6-4cf6-a12f-bdf090e2ad09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d81a060457a4778806d9a4729384bf1, user_id=baf46b1f78bf46b5ad632b347d4fb8ea, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:01.150439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7f4135-b3e6-4cf6-a12f-bdf090e2ad09 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:01.150988 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a0baa5d8-aca3-4272-b1a4-58252bc03f3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73f8d4f76dc246388576b6c863116b24'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:01.164101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1052/2109] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:01 2022] PUT /identity/v3/projects/1d81a060457a4778806d9a4729384bf1/users/baf46b1f78bf46b5ad632b347d4fb8ea/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:02:01.176199 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b256c3f-d397-4fee-a612-9138fd43b90f None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:01.176371 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b256c3f-d397-4fee-a612-9138fd43b90f None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:01.176572 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b256c3f-d397-4fee-a612-9138fd43b90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:01.177113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a0baa5d8-aca3-4272-b1a4-58252bc03f3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73f8d4f76dc246388576b6c863116b24'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:01.178689 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a0baa5d8-aca3-4272-b1a4-58252bc03f3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73f8d4f76dc246388576b6c863116b24'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:01.184262 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:01.194236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:01.194434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a0baa5d8-aca3-4272-b1a4-58252bc03f3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73f8d4f76dc246388576b6c863116b24'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:01.196239 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a0baa5d8-aca3-4272-b1a4-58252bc03f3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73f8d4f76dc246388576b6c863116b24'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:01.214100 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-4b256c3f-d397-4fee-a612-9138fd43b90f None None] MFA Rules not processed for user `baf46b1f78bf46b5ad632b347d4fb8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:01.215963 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-4b256c3f-d397-4fee-a612-9138fd43b90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:01.224112 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1058/2110] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:02:00 2022] DELETE /identity/v3/users/73f8d4f76dc246388576b6c863116b24 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:02:01.316579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1053/2111] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:01 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:01.329409 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0baa5d8-aca3-4272-b1a4-58252bc03f3e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:01.331074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a0baa5d8-aca3-4272-b1a4-58252bc03f3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:01.353464 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b256c3f-d397-4fee-a612-9138fd43b90f None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:01.359597 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-4b256c3f-d397-4fee-a612-9138fd43b90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:01.463478 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0baa5d8-aca3-4272-b1a4-58252bc03f3e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:01.468220 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca769826-da00-4c09-a64d-0fba052aa439 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:01.469835 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca769826-da00-4c09-a64d-0fba052aa439 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:01.470069 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca769826-da00-4c09-a64d-0fba052aa439 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:01.470384 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca769826-da00-4c09-a64d-0fba052aa439 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:01.513052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b256c3f-d397-4fee-a612-9138fd43b90f None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:01.560553 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca769826-da00-4c09-a64d-0fba052aa439 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:01.562101 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca769826-da00-4c09-a64d-0fba052aa439 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:01.584064 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1059/2112] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:01 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:02:01.610860 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ef198-abbf-45ca-9824-66cc8578fff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:01.611846 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ef198-abbf-45ca-9824-66cc8578fff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:01.612350 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ef198-abbf-45ca-9824-66cc8578fff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:01.612664 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ef198-abbf-45ca-9824-66cc8578fff8 admin admin] PATH_INFO: `/v3/projects/1e080f38798142a28c6afa3aa528ef9f` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:01.614236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ef198-abbf-45ca-9824-66cc8578fff8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e080f38798142a28c6afa3aa528ef9f)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:01.616293 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ef198-abbf-45ca-9824-66cc8578fff8 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:01.625984 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca769826-da00-4c09-a64d-0fba052aa439 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:01.627436 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ca769826-da00-4c09-a64d-0fba052aa439 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:01.643243 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca769826-da00-4c09-a64d-0fba052aa439 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:01.657418 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218de2b-bfd0-4347-bdd7-ae658bb16fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:01.658264 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218de2b-bfd0-4347-bdd7-ae658bb16fbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:01.658716 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218de2b-bfd0-4347-bdd7-ae658bb16fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:01.659061 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218de2b-bfd0-4347-bdd7-ae658bb16fbb admin admin] PATH_INFO: `/v3/projects/a6f8618139524e5d95685be35349b6da` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:01.660776 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1218de2b-bfd0-4347-bdd7-ae658bb16fbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6f8618139524e5d95685be35349b6da)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:01.663346 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1218de2b-bfd0-4347-bdd7-ae658bb16fbb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:01.686600 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:01.694618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f53ef198-abbf-45ca-9824-66cc8578fff8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e080f38798142a28c6afa3aa528ef9f', 'request_id': 'req-f53ef198-abbf-45ca-9824-66cc8578fff8'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:01.702394 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f53ef198-abbf-45ca-9824-66cc8578fff8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e080f38798142a28c6afa3aa528ef9f', 'request_id': 'req-f53ef198-abbf-45ca-9824-66cc8578fff8'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:01.709184 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f53ef198-abbf-45ca-9824-66cc8578fff8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e080f38798142a28c6afa3aa528ef9f', 'request_id': 'req-f53ef198-abbf-45ca-9824-66cc8578fff8'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:01.717841 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1218de2b-bfd0-4347-bdd7-ae658bb16fbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6f8618139524e5d95685be35349b6da', 'request_id': 'req-1218de2b-bfd0-4347-bdd7-ae658bb16fbb'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:01.720329 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f53ef198-abbf-45ca-9824-66cc8578fff8 admin admin] The token cache is being invalidate because project 1e080f38798142a28c6afa3aa528ef9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:01.720623 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f53ef198-abbf-45ca-9824-66cc8578fff8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:01.721268 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f53ef198-abbf-45ca-9824-66cc8578fff8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:01.722813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1054/2113] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:01 2022] DELETE /identity/v3/projects/1e080f38798142a28c6afa3aa528ef9f => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:02:01.726195 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ef198-abbf-45ca-9824-66cc8578fff8 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:01.726310 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1218de2b-bfd0-4347-bdd7-ae658bb16fbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6f8618139524e5d95685be35349b6da', 'request_id': 'req-1218de2b-bfd0-4347-bdd7-ae658bb16fbb'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:01.726398 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-f53ef198-abbf-45ca-9824-66cc8578fff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:01.733315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1218de2b-bfd0-4347-bdd7-ae658bb16fbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6f8618139524e5d95685be35349b6da', 'request_id': 'req-1218de2b-bfd0-4347-bdd7-ae658bb16fbb'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:01.744814 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1218de2b-bfd0-4347-bdd7-ae658bb16fbb admin admin] The token cache is being invalidate because project a6f8618139524e5d95685be35349b6da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:01.745193 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1218de2b-bfd0-4347-bdd7-ae658bb16fbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:01.745971 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1218de2b-bfd0-4347-bdd7-ae658bb16fbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:01.747762 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1060/2114] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:01 2022] DELETE /identity/v3/projects/a6f8618139524e5d95685be35349b6da => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:02:01.748932 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85aaaa-3891-4840-9214-a69aaa2b2fe5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:01.749334 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85aaaa-3891-4840-9214-a69aaa2b2fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:01.749611 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85aaaa-3891-4840-9214-a69aaa2b2fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:01.769147 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2a85aaaa-3891-4840-9214-a69aaa2b2fe5 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:01.770364 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2a85aaaa-3891-4840-9214-a69aaa2b2fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:01.774852 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:01.792855 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ef198-abbf-45ca-9824-66cc8578fff8 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:01.830763 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1061/2115] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:01.832137 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0104e7d-331b-4614-ab8b-264318116775 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:01.832434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0104e7d-331b-4614-ab8b-264318116775 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:01.832716 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0104e7d-331b-4614-ab8b-264318116775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:01.842402 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7a824-c0cb-47c6-98aa-52eb2f04d338 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:01.843028 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7a824-c0cb-47c6-98aa-52eb2f04d338 service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:01.843309 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7a824-c0cb-47c6-98aa-52eb2f04d338 service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:01.843616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7a824-c0cb-47c6-98aa-52eb2f04d338 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:01.845476 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-b7f7a824-c0cb-47c6-98aa-52eb2f04d338 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:01.852370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c0104e7d-331b-4614-ab8b-264318116775 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:01.853184 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c0104e7d-331b-4614-ab8b-264318116775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:01.861280 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1062/2116] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:01.862483 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079eec1-2e31-485a-bcd5-038dd299526f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:01.862769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079eec1-2e31-485a-bcd5-038dd299526f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:01.863049 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079eec1-2e31-485a-bcd5-038dd299526f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:01.883457 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5079eec1-2e31-485a-bcd5-038dd299526f None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:01.884709 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5079eec1-2e31-485a-bcd5-038dd299526f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:01.893735 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1063/2117] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:01.898476 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedc88d-a8f7-4287-8746-3b6961b7e719 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:01.898758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedc88d-a8f7-4287-8746-3b6961b7e719 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:01.899031 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedc88d-a8f7-4287-8746-3b6961b7e719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:01.910452 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7a824-c0cb-47c6-98aa-52eb2f04d338 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:01.911961 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7a824-c0cb-47c6-98aa-52eb2f04d338 service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:01.917980 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7aedc88d-a8f7-4287-8746-3b6961b7e719 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:01.919178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7aedc88d-a8f7-4287-8746-3b6961b7e719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:01.921204 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1055/2118] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:02:01.922100 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7a824-c0cb-47c6-98aa-52eb2f04d338 service cinder] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:01.927799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1064/2119] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:01.932281 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aedc88d-a8f7-4287-8746-3b6961b7e719 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:01.934007 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7a824-c0cb-47c6-98aa-52eb2f04d338 service cinder] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:01.941878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16adc1-6345-4380-952b-c84d918b4810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:01.942401 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aedc88d-a8f7-4287-8746-3b6961b7e719 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:01.942571 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16adc1-6345-4380-952b-c84d918b4810 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:01.943155 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16adc1-6345-4380-952b-c84d918b4810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:01.943444 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16adc1-6345-4380-952b-c84d918b4810 admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:01.945072 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a16adc1-6345-4380-952b-c84d918b4810 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:02:01.945387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a16adc1-6345-4380-952b-c84d918b4810 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:01.947353 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a16adc1-6345-4380-952b-c84d918b4810 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:01.947863 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5143eeb-ce47-49ad-8923-a69e359716f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:01.948879 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5143eeb-ce47-49ad-8923-a69e359716f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:01.949167 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5143eeb-ce47-49ad-8923-a69e359716f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:01.949661 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5143eeb-ce47-49ad-8923-a69e359716f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:01.950995 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5143eeb-ce47-49ad-8923-a69e359716f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:02:01.951312 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5143eeb-ce47-49ad-8923-a69e359716f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:01.952990 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5143eeb-ce47-49ad-8923-a69e359716f7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:01.960754 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1056/2120] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:02:01.965273 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16adc1-6345-4380-952b-c84d918b4810 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:01.967595 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1065/2121] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 05:02:01.971065 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5143eeb-ce47-49ad-8923-a69e359716f7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:01.975982 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16adc1-6345-4380-952b-c84d918b4810 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:01.980971 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81acc524-7ae4-42af-b177-54e7285996cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:01.981302 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acc524-7ae4-42af-b177-54e7285996cb admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:01.981746 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acc524-7ae4-42af-b177-54e7285996cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:01.982043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acc524-7ae4-42af-b177-54e7285996cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:01.982829 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81acc524-7ae4-42af-b177-54e7285996cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:01.983546 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5143eeb-ce47-49ad-8923-a69e359716f7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:01.985790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81acc524-7ae4-42af-b177-54e7285996cb admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:01.990587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa65f3-f667-45dc-92a2-f6b0694957f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:01.991572 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa65f3-f667-45dc-92a2-f6b0694957f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:01.992238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa65f3-f667-45dc-92a2-f6b0694957f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:01.992638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa65f3-f667-45dc-92a2-f6b0694957f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:01.993699 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa65f3-f667-45dc-92a2-f6b0694957f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:01.995427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa65f3-f667-45dc-92a2-f6b0694957f5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:02.018681 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1057/2122] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:02:01 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:02:02.023412 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81acc524-7ae4-42af-b177-54e7285996cb admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:02.028898 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1066/2123] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:02:01 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:02.030518 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:02.033789 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa65f3-f667-45dc-92a2-f6b0694957f5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:02.109314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81acc524-7ae4-42af-b177-54e7285996cb admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:02.143363 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa65f3-f667-45dc-92a2-f6b0694957f5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:02.148676 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb05c3-cc24-4f3f-8e9c-7c568169ec96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:02.149324 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb05c3-cc24-4f3f-8e9c-7c568169ec96 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:02.149771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb05c3-cc24-4f3f-8e9c-7c568169ec96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:02.150063 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb05c3-cc24-4f3f-8e9c-7c568169ec96 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:02.150792 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb05c3-cc24-4f3f-8e9c-7c568169ec96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:02.152778 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb05c3-cc24-4f3f-8e9c-7c568169ec96 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:02.174100 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7452d4-cbce-476d-a709-adca830a666e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:02.174751 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7452d4-cbce-476d-a709-adca830a666e admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:02.175034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7452d4-cbce-476d-a709-adca830a666e admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:02.175300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7452d4-cbce-476d-a709-adca830a666e admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:02.176002 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7452d4-cbce-476d-a709-adca830a666e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:02.177996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7452d4-cbce-476d-a709-adca830a666e admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:02.182198 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1067/2124] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:02:02 2022] POST /identity/v3/users => generated 288 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:02:02.183193 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb05c3-cc24-4f3f-8e9c-7c568169ec96 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:02.193812 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb05c3-cc24-4f3f-8e9c-7c568169ec96 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:02.199687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af682c29-84bc-4a31-b1d5-e6614c8c8090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:02.200327 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af682c29-84bc-4a31-b1d5-e6614c8c8090 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:02.200615 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af682c29-84bc-4a31-b1d5-e6614c8c8090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:02.200973 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af682c29-84bc-4a31-b1d5-e6614c8c8090 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:02.201714 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af682c29-84bc-4a31-b1d5-e6614c8c8090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:02.201952 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1058/2125] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:02:02 2022] POST /identity/v3/users => generated 283 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:02:02.203455 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af682c29-84bc-4a31-b1d5-e6614c8c8090 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:02.206248 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7452d4-cbce-476d-a709-adca830a666e admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:02.215302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1068/2126] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:02.216638 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7452d4-cbce-476d-a709-adca830a666e admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:02.219368 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af682c29-84bc-4a31-b1d5-e6614c8c8090 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:02.222319 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c3b12-750b-4170-a7da-668c7f9996a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:02.223108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c3b12-750b-4170-a7da-668c7f9996a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:02.223462 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c3b12-750b-4170-a7da-668c7f9996a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:02.223900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c3b12-750b-4170-a7da-668c7f9996a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:02.224848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c3b12-750b-4170-a7da-668c7f9996a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:02.227324 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c3b12-750b-4170-a7da-668c7f9996a5 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:02.230082 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af682c29-84bc-4a31-b1d5-e6614c8c8090 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:02.235715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad49ed-fd87-4c9c-8d61-97b627a4e294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:02.236370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad49ed-fd87-4c9c-8d61-97b627a4e294 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:02.236688 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad49ed-fd87-4c9c-8d61-97b627a4e294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:02.236940 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad49ed-fd87-4c9c-8d61-97b627a4e294 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:02.237734 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad49ed-fd87-4c9c-8d61-97b627a4e294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:02.238051 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1059/2127] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:02:02.241160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad49ed-fd87-4c9c-8d61-97b627a4e294 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:02.242477 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c3b12-750b-4170-a7da-668c7f9996a5 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:02.251998 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1069/2128] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:02:02.252776 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c3b12-750b-4170-a7da-668c7f9996a5 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:02.256159 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad49ed-fd87-4c9c-8d61-97b627a4e294 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:02.257606 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3017b1-99fb-421e-a097-8f7d5dbe6967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:02.258350 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3017b1-99fb-421e-a097-8f7d5dbe6967 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:02.258523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3017b1-99fb-421e-a097-8f7d5dbe6967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:02.258799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3017b1-99fb-421e-a097-8f7d5dbe6967 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:02.259520 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3017b1-99fb-421e-a097-8f7d5dbe6967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:02.261190 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3017b1-99fb-421e-a097-8f7d5dbe6967 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:02.267104 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad49ed-fd87-4c9c-8d61-97b627a4e294 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:02.273011 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1060/2129] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:02:02.273145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeccd802-c9b8-4d01-8fca-1ce990749ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:02.276407 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeccd802-c9b8-4d01-8fca-1ce990749ccb admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:02.276407 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeccd802-c9b8-4d01-8fca-1ce990749ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:02.276407 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeccd802-c9b8-4d01-8fca-1ce990749ccb admin admin] PATH_INFO: `/v3/projects/7b90b49d3f014f7baed569a0f9a1d1fa/users/c3c8f928717c4a03816d70bbee64d1c0/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:02.277574 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3017b1-99fb-421e-a097-8f7d5dbe6967 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:02.289693 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3017b1-99fb-421e-a097-8f7d5dbe6967 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:02.290837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeccd802-c9b8-4d01-8fca-1ce990749ccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b90b49d3f014f7baed569a0f9a1d1fa, user_id=c3c8f928717c4a03816d70bbee64d1c0, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:02.293823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeccd802-c9b8-4d01-8fca-1ce990749ccb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:02.294307 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9af4b-8240-4596-90d8-0e3f9d94425a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:02.294980 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9af4b-8240-4596-90d8-0e3f9d94425a admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:02.295257 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9af4b-8240-4596-90d8-0e3f9d94425a admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:02.295992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9af4b-8240-4596-90d8-0e3f9d94425a admin admin] PATH_INFO: `/v3/projects/b43e812e5c254c13be16ba66a616a79e/users/857f89085c794bd1b746f74675bd246f/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:02.305289 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:02.311383 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1070/2130] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:02 2022] PUT /identity/v3/projects/7b90b49d3f014f7baed569a0f9a1d1fa/users/c3c8f928717c4a03816d70bbee64d1c0/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:02:02.324633 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de344fa-f2ea-4a27-8fa4-e7533cf016eb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:02.325087 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de344fa-f2ea-4a27-8fa4-e7533cf016eb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:02.325427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de344fa-f2ea-4a27-8fa4-e7533cf016eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:02.329690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9af4b-8240-4596-90d8-0e3f9d94425a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b43e812e5c254c13be16ba66a616a79e, user_id=857f89085c794bd1b746f74675bd246f, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:02.332678 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9af4b-8240-4596-90d8-0e3f9d94425a admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:02.349864 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1061/2131] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:02 2022] PUT /identity/v3/projects/b43e812e5c254c13be16ba66a616a79e/users/857f89085c794bd1b746f74675bd246f/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:02.358132 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1de344fa-f2ea-4a27-8fa4-e7533cf016eb None None] MFA Rules not processed for user `c3c8f928717c4a03816d70bbee64d1c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:02.359464 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1de344fa-f2ea-4a27-8fa4-e7533cf016eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:02.361578 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17462f-50c1-42fe-b2ff-46a0bd871ffa None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:02.361873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17462f-50c1-42fe-b2ff-46a0bd871ffa None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:02.362156 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17462f-50c1-42fe-b2ff-46a0bd871ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:02.395058 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-7e17462f-50c1-42fe-b2ff-46a0bd871ffa None None] MFA Rules not processed for user `857f89085c794bd1b746f74675bd246f`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:02.396587 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-7e17462f-50c1-42fe-b2ff-46a0bd871ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:02.449312 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1071/2132] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:02:02.450612 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:02.453954 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de344fa-f2ea-4a27-8fa4-e7533cf016eb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:02.480133 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1062/2133] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:02 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:02:02.484442 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17462f-50c1-42fe-b2ff-46a0bd871ffa None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:02.531485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de344fa-f2ea-4a27-8fa4-e7533cf016eb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:02.536535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35739d8e-fd00-4a05-9e0e-2f58414f40ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:02.537179 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35739d8e-fd00-4a05-9e0e-2f58414f40ed admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:02.537474 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35739d8e-fd00-4a05-9e0e-2f58414f40ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:02.537744 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35739d8e-fd00-4a05-9e0e-2f58414f40ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:02.538461 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35739d8e-fd00-4a05-9e0e-2f58414f40ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:02.540529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35739d8e-fd00-4a05-9e0e-2f58414f40ed admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:02.551529 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17462f-50c1-42fe-b2ff-46a0bd871ffa None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:02.556519 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784cf18e-ea3b-46ec-ba54-4ada26f68efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:02.557289 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784cf18e-ea3b-46ec-ba54-4ada26f68efd admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:02.557579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784cf18e-ea3b-46ec-ba54-4ada26f68efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:02.557860 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784cf18e-ea3b-46ec-ba54-4ada26f68efd admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:02.558570 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784cf18e-ea3b-46ec-ba54-4ada26f68efd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:02.560592 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784cf18e-ea3b-46ec-ba54-4ada26f68efd admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:02.573864 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1072/2134] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:02:02 2022] POST /identity/v3/projects => generated 377 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:02.578112 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35739d8e-fd00-4a05-9e0e-2f58414f40ed admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:02.592308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1063/2135] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:02:02 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:02:02.593561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784cf18e-ea3b-46ec-ba54-4ada26f68efd admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:02.642874 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35739d8e-fd00-4a05-9e0e-2f58414f40ed admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:02.662436 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:02.662934 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784cf18e-ea3b-46ec-ba54-4ada26f68efd admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:02.668382 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cd748-685d-4f9f-aa6c-661ab7d73923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:02.669044 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cd748-685d-4f9f-aa6c-661ab7d73923 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:02.669334 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cd748-685d-4f9f-aa6c-661ab7d73923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:02.669618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cd748-685d-4f9f-aa6c-661ab7d73923 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:02.670293 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733cd748-685d-4f9f-aa6c-661ab7d73923 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:02.672708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733cd748-685d-4f9f-aa6c-661ab7d73923 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:02.704411 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1064/2136] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:02:02 2022] POST /identity/v3/users => generated 289 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:02:02.708675 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cd748-685d-4f9f-aa6c-661ab7d73923 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:02.720370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e38206-75ea-4b43-804a-8d336a0a80d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:02.721052 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e38206-75ea-4b43-804a-8d336a0a80d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:02.721343 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e38206-75ea-4b43-804a-8d336a0a80d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:02.721626 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e38206-75ea-4b43-804a-8d336a0a80d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:02.722103 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cd748-685d-4f9f-aa6c-661ab7d73923 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:02.722307 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e38206-75ea-4b43-804a-8d336a0a80d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:02.724300 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e38206-75ea-4b43-804a-8d336a0a80d1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:02.727633 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518dcc2-8c5e-4d8d-8ad5-ca34cf1a5e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:02.728419 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518dcc2-8c5e-4d8d-8ad5-ca34cf1a5e06 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:02.728700 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518dcc2-8c5e-4d8d-8ad5-ca34cf1a5e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:02.728978 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518dcc2-8c5e-4d8d-8ad5-ca34cf1a5e06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:02.730161 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a518dcc2-8c5e-4d8d-8ad5-ca34cf1a5e06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:02.732137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a518dcc2-8c5e-4d8d-8ad5-ca34cf1a5e06 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:02.748848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1065/2137] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:02.753109 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518dcc2-8c5e-4d8d-8ad5-ca34cf1a5e06 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:02.761316 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1073/2138] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:02:02 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:02:02.763276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e38206-75ea-4b43-804a-8d336a0a80d1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:02.765327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518dcc2-8c5e-4d8d-8ad5-ca34cf1a5e06 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:02.770145 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39b6f2-5adf-4cb6-bb80-213843dca055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:02.770909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39b6f2-5adf-4cb6-bb80-213843dca055 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:02.771245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39b6f2-5adf-4cb6-bb80-213843dca055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:02.771523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39b6f2-5adf-4cb6-bb80-213843dca055 admin admin] PATH_INFO: `/v3/projects/4918431a1cc34242a77903d241406bf4/users/cc3ed6a7330f44e18fd480d317fd243a/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:02.773602 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e38206-75ea-4b43-804a-8d336a0a80d1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:02.778320 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3383cdc2-680c-4918-af0d-6139499057c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:02.778947 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383cdc2-680c-4918-af0d-6139499057c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:02.779233 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383cdc2-680c-4918-af0d-6139499057c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:02.779511 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383cdc2-680c-4918-af0d-6139499057c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:02.780277 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3383cdc2-680c-4918-af0d-6139499057c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:02.782114 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3383cdc2-680c-4918-af0d-6139499057c0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:02.787792 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39b6f2-5adf-4cb6-bb80-213843dca055 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4918431a1cc34242a77903d241406bf4, user_id=cc3ed6a7330f44e18fd480d317fd243a, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:02.790665 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39b6f2-5adf-4cb6-bb80-213843dca055 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:02.792820 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1074/2139] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:02.796896 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3383cdc2-680c-4918-af0d-6139499057c0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:02.804906 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1066/2140] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:02 2022] PUT /identity/v3/projects/4918431a1cc34242a77903d241406bf4/users/cc3ed6a7330f44e18fd480d317fd243a/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:02.807870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3383cdc2-680c-4918-af0d-6139499057c0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:02.808872 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39b6f2-5adf-4cb6-bb80-213843dca055 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:02.836015 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:02.866355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a000b6d-3c31-4fd4-907a-788c694b9a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:02.866996 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a000b6d-3c31-4fd4-907a-788c694b9a9f admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:02.867282 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a000b6d-3c31-4fd4-907a-788c694b9a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:02.867589 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a000b6d-3c31-4fd4-907a-788c694b9a9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:02.868283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a000b6d-3c31-4fd4-907a-788c694b9a9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:02.869895 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a000b6d-3c31-4fd4-907a-788c694b9a9f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:02.873919 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39b6f2-5adf-4cb6-bb80-213843dca055 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:02.879055 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d589a5a-58c9-41b9-87cc-6144c8ea263c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:02.879707 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d589a5a-58c9-41b9-87cc-6144c8ea263c admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:02.879851 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1075/2141] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:02:02.880860 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d589a5a-58c9-41b9-87cc-6144c8ea263c admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:02.881164 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d589a5a-58c9-41b9-87cc-6144c8ea263c admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:02.881918 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d589a5a-58c9-41b9-87cc-6144c8ea263c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:02.883689 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d589a5a-58c9-41b9-87cc-6144c8ea263c admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:02.884002 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a000b6d-3c31-4fd4-907a-788c694b9a9f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:02.894162 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a000b6d-3c31-4fd4-907a-788c694b9a9f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:02.894659 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1067/2142] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:02:02.898981 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d589a5a-58c9-41b9-87cc-6144c8ea263c admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:02.899851 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e89a3-5731-4180-bd7a-fc7204bf45bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:02.900590 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e89a3-5731-4180-bd7a-fc7204bf45bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:02.900951 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e89a3-5731-4180-bd7a-fc7204bf45bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:02.901297 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e89a3-5731-4180-bd7a-fc7204bf45bb admin admin] PATH_INFO: `/v3/projects/95f9eae1576f47ebb786c92a32f7fa24/users/57e500a5809547d39b1c0d7342d37920/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:02.910844 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d589a5a-58c9-41b9-87cc-6144c8ea263c admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:02.916394 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460b4d7-567e-4737-ad7f-3be8a1268595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:02.917121 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460b4d7-567e-4737-ad7f-3be8a1268595 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:02.917410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460b4d7-567e-4737-ad7f-3be8a1268595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:02.917692 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460b4d7-567e-4737-ad7f-3be8a1268595 admin admin] PATH_INFO: `/v3/projects/4918431a1cc34242a77903d241406bf4/users/cc3ed6a7330f44e18fd480d317fd243a/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:02.919700 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460b4d7-567e-4737-ad7f-3be8a1268595 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4918431a1cc34242a77903d241406bf4, user_id=cc3ed6a7330f44e18fd480d317fd243a, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:02.921555 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e89a3-5731-4180-bd7a-fc7204bf45bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=95f9eae1576f47ebb786c92a32f7fa24, user_id=57e500a5809547d39b1c0d7342d37920, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:02.922399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460b4d7-567e-4737-ad7f-3be8a1268595 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:02.924418 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e89a3-5731-4180-bd7a-fc7204bf45bb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:02.934383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1068/2143] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:02 2022] PUT /identity/v3/projects/4918431a1cc34242a77903d241406bf4/users/cc3ed6a7330f44e18fd480d317fd243a/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:02:02.939688 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1076/2144] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:02 2022] PUT /identity/v3/projects/95f9eae1576f47ebb786c92a32f7fa24/users/57e500a5809547d39b1c0d7342d37920/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:02:02.945625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d3686-8e8b-4e3e-818f-1234b9049d3d None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:02.945858 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d3686-8e8b-4e3e-818f-1234b9049d3d None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:02.946134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d3686-8e8b-4e3e-818f-1234b9049d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:02.950730 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23441e-2cfa-483f-97e8-6fb4b97e6b2b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:02.951163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23441e-2cfa-483f-97e8-6fb4b97e6b2b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:02.951537 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23441e-2cfa-483f-97e8-6fb4b97e6b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:02.979352 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-1e3d3686-8e8b-4e3e-818f-1234b9049d3d None None] MFA Rules not processed for user `cc3ed6a7330f44e18fd480d317fd243a`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:02.979928 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ec23441e-2cfa-483f-97e8-6fb4b97e6b2b None None] MFA Rules not processed for user `57e500a5809547d39b1c0d7342d37920`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:02.981171 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ec23441e-2cfa-483f-97e8-6fb4b97e6b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:02.981391 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-1e3d3686-8e8b-4e3e-818f-1234b9049d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:03.045214 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:03.049445 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1077/2145] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:03.053905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23441e-2cfa-483f-97e8-6fb4b97e6b2b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:03.059872 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1069/2146] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:02 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:02:03.073837 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d3686-8e8b-4e3e-818f-1234b9049d3d None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:03.075359 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-1e3d3686-8e8b-4e3e-818f-1234b9049d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:03.126390 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23441e-2cfa-483f-97e8-6fb4b97e6b2b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:03.132164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d646168-2e17-41e3-b1c2-957e7e1b3c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:03.133528 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d646168-2e17-41e3-b1c2-957e7e1b3c47 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:03.133923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d646168-2e17-41e3-b1c2-957e7e1b3c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:03.134168 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d646168-2e17-41e3-b1c2-957e7e1b3c47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:03.134840 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d646168-2e17-41e3-b1c2-957e7e1b3c47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:03.136796 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d646168-2e17-41e3-b1c2-957e7e1b3c47 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:03.144199 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d3686-8e8b-4e3e-818f-1234b9049d3d None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:03.148409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e0a4d3-1f09-470d-aef6-a7e384c31bcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:03.149439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0a4d3-1f09-470d-aef6-a7e384c31bcc service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:03.149729 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0a4d3-1f09-470d-aef6-a7e384c31bcc service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:03.150006 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0a4d3-1f09-470d-aef6-a7e384c31bcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:03.166100 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1078/2147] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:02:03 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:03.169379 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e0a4d3-1f09-470d-aef6-a7e384c31bcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:03.170256 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d646168-2e17-41e3-b1c2-957e7e1b3c47 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:03.171181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e0a4d3-1f09-470d-aef6-a7e384c31bcc service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:03.187211 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1070/2148] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:03 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:02:03.247844 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d646168-2e17-41e3-b1c2-957e7e1b3c47 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:03.252657 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b801a-d6d3-4d80-bd3a-20873e13719f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:03.253319 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b801a-d6d3-4d80-bd3a-20873e13719f admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:03.253606 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b801a-d6d3-4d80-bd3a-20873e13719f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:03.253892 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b801a-d6d3-4d80-bd3a-20873e13719f admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:03.254564 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b801a-d6d3-4d80-bd3a-20873e13719f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:03.256538 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b801a-d6d3-4d80-bd3a-20873e13719f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:03.280072 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1079/2149] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:02:03 2022] POST /identity/v3/users => generated 295 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:02:03.284537 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e0a4d3-1f09-470d-aef6-a7e384c31bcc service cinder] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:03.292289 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b801a-d6d3-4d80-bd3a-20873e13719f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:03.296024 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e0a4d3-1f09-470d-aef6-a7e384c31bcc service cinder] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:03.302091 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd42f7-dc0f-4786-a31c-bb5c299c0755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:03.302785 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd42f7-dc0f-4786-a31c-bb5c299c0755 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:03.304272 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd42f7-dc0f-4786-a31c-bb5c299c0755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:03.304566 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd42f7-dc0f-4786-a31c-bb5c299c0755 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:03.305427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd42f7-dc0f-4786-a31c-bb5c299c0755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:03.307166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd42f7-dc0f-4786-a31c-bb5c299c0755 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:03.317958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1071/2150] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:02:03.322172 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd42f7-dc0f-4786-a31c-bb5c299c0755 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:03.336753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd42f7-dc0f-4786-a31c-bb5c299c0755 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:03.343325 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82a6e6-6144-46da-a04e-4c4f4cf1f81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:03.344055 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82a6e6-6144-46da-a04e-4c4f4cf1f81d admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:03.344341 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82a6e6-6144-46da-a04e-4c4f4cf1f81d admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:03.344620 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82a6e6-6144-46da-a04e-4c4f4cf1f81d admin admin] PATH_INFO: `/v3/projects/0a369aad0b444cfcbfad28d50efa1a3a/users/e693eb8b5dd84f5ab02cf3956ca2ff76/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:03.362876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b801a-d6d3-4d80-bd3a-20873e13719f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:03.363125 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f82a6e6-6144-46da-a04e-4c4f4cf1f81d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a369aad0b444cfcbfad28d50efa1a3a, user_id=e693eb8b5dd84f5ab02cf3956ca2ff76, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:03.366579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f82a6e6-6144-46da-a04e-4c4f4cf1f81d admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:03.367395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317f3c3-1edf-403b-bbf4-d6b180fc07da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:03.368132 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317f3c3-1edf-403b-bbf4-d6b180fc07da service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:03.368416 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317f3c3-1edf-403b-bbf4-d6b180fc07da service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:03.368700 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317f3c3-1edf-403b-bbf4-d6b180fc07da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:03.378248 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:03.386624 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1072/2151] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:03 2022] PUT /identity/v3/projects/0a369aad0b444cfcbfad28d50efa1a3a/users/e693eb8b5dd84f5ab02cf3956ca2ff76/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:02:03.387500 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82a6e6-6144-46da-a04e-4c4f4cf1f81d admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:03.412546 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:03.446433 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9317f3c3-1edf-403b-bbf4-d6b180fc07da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:03.448983 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9317f3c3-1edf-403b-bbf4-d6b180fc07da service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:03.459492 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1080/2152] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:03 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:02:03.462841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82a6e6-6144-46da-a04e-4c4f4cf1f81d admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:03.468018 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a4e284-52d3-439e-ba0c-e2a5b7b9cbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:03.468757 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4e284-52d3-439e-ba0c-e2a5b7b9cbb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:03.469622 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4e284-52d3-439e-ba0c-e2a5b7b9cbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:03.469622 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4e284-52d3-439e-ba0c-e2a5b7b9cbb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:03.480561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a4e284-52d3-439e-ba0c-e2a5b7b9cbb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:03.482662 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a4e284-52d3-439e-ba0c-e2a5b7b9cbb9 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:03.502837 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1073/2153] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:03.507293 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317f3c3-1edf-403b-bbf4-d6b180fc07da service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:03.526815 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317f3c3-1edf-403b-bbf4-d6b180fc07da service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:03.532515 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965739c2-e844-492e-a2af-29f3811b2215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:03.533301 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965739c2-e844-492e-a2af-29f3811b2215 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:03.533508 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965739c2-e844-492e-a2af-29f3811b2215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:03.533790 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965739c2-e844-492e-a2af-29f3811b2215 admin admin] PATH_INFO: `/v3/projects/0a369aad0b444cfcbfad28d50efa1a3a/users/e693eb8b5dd84f5ab02cf3956ca2ff76/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:03.535927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965739c2-e844-492e-a2af-29f3811b2215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a369aad0b444cfcbfad28d50efa1a3a, user_id=e693eb8b5dd84f5ab02cf3956ca2ff76, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:03.538872 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965739c2-e844-492e-a2af-29f3811b2215 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:03.551020 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1081/2154] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:03 2022] PUT /identity/v3/projects/0a369aad0b444cfcbfad28d50efa1a3a/users/e693eb8b5dd84f5ab02cf3956ca2ff76/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:03.565103 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958b80e-2055-4159-a098-39599741c1de None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:03.565435 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958b80e-2055-4159-a098-39599741c1de None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:03.565715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958b80e-2055-4159-a098-39599741c1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:03.617595 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-7958b80e-2055-4159-a098-39599741c1de None None] MFA Rules not processed for user `e693eb8b5dd84f5ab02cf3956ca2ff76`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:03.618905 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-7958b80e-2055-4159-a098-39599741c1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:03.704407 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1074/2155] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:03 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:02:03.783305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965739c2-e844-492e-a2af-29f3811b2215 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:03.808005 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958b80e-2055-4159-a098-39599741c1de None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:03.809542 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-7958b80e-2055-4159-a098-39599741c1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:03.861600 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965739c2-e844-492e-a2af-29f3811b2215 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:03.865463 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28effa96-7b7c-4d6a-b28a-b6710db0144a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:03.866107 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28effa96-7b7c-4d6a-b28a-b6710db0144a service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:03.866481 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28effa96-7b7c-4d6a-b28a-b6710db0144a service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:03.867000 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28effa96-7b7c-4d6a-b28a-b6710db0144a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:03.899816 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958b80e-2055-4159-a098-39599741c1de None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:03.904596 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b23899-b1bf-4b1e-8328-08eb4bbf0f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:03.905260 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b23899-b1bf-4b1e-8328-08eb4bbf0f0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:03.905805 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b23899-b1bf-4b1e-8328-08eb4bbf0f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:03.905805 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b23899-b1bf-4b1e-8328-08eb4bbf0f0f admin admin] PATH_INFO: `/v3/users/b7585a95bd4e48aa9e3d8dce5845e3c4` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:03.907035 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b23899-b1bf-4b1e-8328-08eb4bbf0f0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7585a95bd4e48aa9e3d8dce5845e3c4)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:03.909114 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b23899-b1bf-4b1e-8328-08eb4bbf0f0f admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:03.935138 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28effa96-7b7c-4d6a-b28a-b6710db0144a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:03.936668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28effa96-7b7c-4d6a-b28a-b6710db0144a service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:03.937517 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:03.947848 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1082/2156] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:02:03.966738 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-a1b23899-b1bf-4b1e-8328-08eb4bbf0f0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7585a95bd4e48aa9e3d8dce5845e3c4'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:03.974291 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-a1b23899-b1bf-4b1e-8328-08eb4bbf0f0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7585a95bd4e48aa9e3d8dce5845e3c4'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:03.982416 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-a1b23899-b1bf-4b1e-8328-08eb4bbf0f0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7585a95bd4e48aa9e3d8dce5845e3c4'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:03.983138 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-a1b23899-b1bf-4b1e-8328-08eb4bbf0f0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7585a95bd4e48aa9e3d8dce5845e3c4'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:03.983838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-a1b23899-b1bf-4b1e-8328-08eb4bbf0f0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7585a95bd4e48aa9e3d8dce5845e3c4'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:04.006420 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1075/2157] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:02:03 2022] DELETE /identity/v3/users/b7585a95bd4e48aa9e3d8dce5845e3c4 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:02:04.011675 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f78b07-8f7f-4fe1-a029-bd46e6f52204 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:04.011965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f78b07-8f7f-4fe1-a029-bd46e6f52204 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:04.012469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f78b07-8f7f-4fe1-a029-bd46e6f52204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:04.033436 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c7f78b07-8f7f-4fe1-a029-bd46e6f52204 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:04.034865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c7f78b07-8f7f-4fe1-a029-bd46e6f52204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:04.096372 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1083/2158] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:04.107151 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b23899-b1bf-4b1e-8328-08eb4bbf0f0f admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:04.108618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a1b23899-b1bf-4b1e-8328-08eb4bbf0f0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:04.171396 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b23899-b1bf-4b1e-8328-08eb4bbf0f0f admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:04.175291 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b51c7b-ee7c-4ddb-b3b3-aa46f95d53b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:04.176014 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b51c7b-ee7c-4ddb-b3b3-aa46f95d53b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:04.176312 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b51c7b-ee7c-4ddb-b3b3-aa46f95d53b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:04.176668 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b51c7b-ee7c-4ddb-b3b3-aa46f95d53b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:04.185934 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b51c7b-ee7c-4ddb-b3b3-aa46f95d53b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:04.187750 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b51c7b-ee7c-4ddb-b3b3-aa46f95d53b5 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:04.198582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1076/2159] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:02:04.528955 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f78b07-8f7f-4fe1-a029-bd46e6f52204 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:04.530445 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c7f78b07-8f7f-4fe1-a029-bd46e6f52204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:04.547307 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f78b07-8f7f-4fe1-a029-bd46e6f52204 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:04.553243 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c42aa3-bf3a-4bd0-a4fd-323cca92811a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:04.553971 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c42aa3-bf3a-4bd0-a4fd-323cca92811a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:04.554255 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c42aa3-bf3a-4bd0-a4fd-323cca92811a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:04.554540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c42aa3-bf3a-4bd0-a4fd-323cca92811a admin admin] PATH_INFO: `/v3/projects/47c1709e6f71447881832a20e70746b0` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:04.556048 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c42aa3-bf3a-4bd0-a4fd-323cca92811a admin admin] RBAC: Authorizing `identity:delete_project(project_id=47c1709e6f71447881832a20e70746b0)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:04.557879 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c42aa3-bf3a-4bd0-a4fd-323cca92811a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:04.565361 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:04.607436 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f2c42aa3-bf3a-4bd0-a4fd-323cca92811a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47c1709e6f71447881832a20e70746b0', 'request_id': 'req-f2c42aa3-bf3a-4bd0-a4fd-323cca92811a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:04.613697 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f2c42aa3-bf3a-4bd0-a4fd-323cca92811a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47c1709e6f71447881832a20e70746b0', 'request_id': 'req-f2c42aa3-bf3a-4bd0-a4fd-323cca92811a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:04.622971 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f2c42aa3-bf3a-4bd0-a4fd-323cca92811a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47c1709e6f71447881832a20e70746b0', 'request_id': 'req-f2c42aa3-bf3a-4bd0-a4fd-323cca92811a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:04.633950 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f2c42aa3-bf3a-4bd0-a4fd-323cca92811a admin admin] The token cache is being invalidate because project 47c1709e6f71447881832a20e70746b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:04.634235 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f2c42aa3-bf3a-4bd0-a4fd-323cca92811a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:04.634853 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f2c42aa3-bf3a-4bd0-a4fd-323cca92811a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:04.636423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1084/2160] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:04 2022] DELETE /identity/v3/projects/47c1709e6f71447881832a20e70746b0 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:04.643009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f36c6-1ece-4bcc-b22b-3887f0725316 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:04.643272 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f36c6-1ece-4bcc-b22b-3887f0725316 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:04.643568 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f36c6-1ece-4bcc-b22b-3887f0725316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:04.666869 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-631f36c6-1ece-4bcc-b22b-3887f0725316 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:04.667428 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-631f36c6-1ece-4bcc-b22b-3887f0725316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:04.730919 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1077/2161] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:04.738543 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8621aa-7685-434b-a88a-7eb216c0a7a2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:04.738860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8621aa-7685-434b-a88a-7eb216c0a7a2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:04.739139 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8621aa-7685-434b-a88a-7eb216c0a7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:04.761205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-eb8621aa-7685-434b-a88a-7eb216c0a7a2 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:04.762530 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-eb8621aa-7685-434b-a88a-7eb216c0a7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:04.770891 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1085/2162] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:04.775324 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f36c6-1ece-4bcc-b22b-3887f0725316 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:04.785277 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f36c6-1ece-4bcc-b22b-3887f0725316 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:04.790501 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec9c8d-10b0-4d17-80c2-3151416069e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:04.791196 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec9c8d-10b0-4d17-80c2-3151416069e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:04.791482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec9c8d-10b0-4d17-80c2-3151416069e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:04.791801 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec9c8d-10b0-4d17-80c2-3151416069e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:04.792516 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec9c8d-10b0-4d17-80c2-3151416069e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:02:04.792824 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec9c8d-10b0-4d17-80c2-3151416069e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:04.794455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec9c8d-10b0-4d17-80c2-3151416069e9 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:04.807615 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1078/2163] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:02:04.812019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8621aa-7685-434b-a88a-7eb216c0a7a2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:04.823772 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8621aa-7685-434b-a88a-7eb216c0a7a2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:04.829946 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac95f13-6bb5-458c-831e-4fea247cf17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:04.830626 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac95f13-6bb5-458c-831e-4fea247cf17d admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:04.831121 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac95f13-6bb5-458c-831e-4fea247cf17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:04.831417 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac95f13-6bb5-458c-831e-4fea247cf17d admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:04.832186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac95f13-6bb5-458c-831e-4fea247cf17d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:04.834511 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac95f13-6bb5-458c-831e-4fea247cf17d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:04.877214 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1086/2164] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:02:04 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:04.877398 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec9c8d-10b0-4d17-80c2-3151416069e9 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:04.889803 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec9c8d-10b0-4d17-80c2-3151416069e9 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:04.933372 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:04.973292 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255be269-20d0-41d1-915c-27884dbc0551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:04.973912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255be269-20d0-41d1-915c-27884dbc0551 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:04.974197 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255be269-20d0-41d1-915c-27884dbc0551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:04.974470 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255be269-20d0-41d1-915c-27884dbc0551 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:04.975150 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255be269-20d0-41d1-915c-27884dbc0551 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:04.977374 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255be269-20d0-41d1-915c-27884dbc0551 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:05.005375 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1079/2165] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:02:04 2022] POST /identity/v3/users => generated 288 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:02:05.009699 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac95f13-6bb5-458c-831e-4fea247cf17d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:05.019926 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac95f13-6bb5-458c-831e-4fea247cf17d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:05.024567 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9ff71-2abb-4b51-b577-091153e5e9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:05.025194 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9ff71-2abb-4b51-b577-091153e5e9ac admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:05.025474 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9ff71-2abb-4b51-b577-091153e5e9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:05.025746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9ff71-2abb-4b51-b577-091153e5e9ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:05.026442 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9ff71-2abb-4b51-b577-091153e5e9ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:05.028081 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9ff71-2abb-4b51-b577-091153e5e9ac admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:05.040521 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1087/2166] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:05.045256 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255be269-20d0-41d1-915c-27884dbc0551 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:05.056499 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255be269-20d0-41d1-915c-27884dbc0551 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:05.061568 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f08526-ebfe-44a3-8452-b26c87314f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:05.062213 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f08526-ebfe-44a3-8452-b26c87314f9c admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:05.062502 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f08526-ebfe-44a3-8452-b26c87314f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:05.062801 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f08526-ebfe-44a3-8452-b26c87314f9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:05.063509 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f08526-ebfe-44a3-8452-b26c87314f9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:05.065180 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f08526-ebfe-44a3-8452-b26c87314f9c admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:05.076249 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1080/2167] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:05.080610 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9ff71-2abb-4b51-b577-091153e5e9ac admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:05.091312 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9ff71-2abb-4b51-b577-091153e5e9ac admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:05.096142 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61722677-ad3e-45c2-8230-74ab90623384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:05.096827 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61722677-ad3e-45c2-8230-74ab90623384 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:05.097108 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61722677-ad3e-45c2-8230-74ab90623384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:05.097386 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61722677-ad3e-45c2-8230-74ab90623384 admin admin] PATH_INFO: `/v3/projects/42652e9232b1477c9342c6d618ba1ad7/users/815705d191904e6f977b68273f59b66a/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:05.114454 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61722677-ad3e-45c2-8230-74ab90623384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42652e9232b1477c9342c6d618ba1ad7, user_id=815705d191904e6f977b68273f59b66a, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:05.117159 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61722677-ad3e-45c2-8230-74ab90623384 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:05.133975 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1088/2168] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:05 2022] PUT /identity/v3/projects/42652e9232b1477c9342c6d618ba1ad7/users/815705d191904e6f977b68273f59b66a/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:02:05.144501 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f91d5-7474-4010-b2be-7766065f12cf None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:05.144824 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f91d5-7474-4010-b2be-7766065f12cf None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:05.145109 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f91d5-7474-4010-b2be-7766065f12cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:05.175463 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-d29f91d5-7474-4010-b2be-7766065f12cf None None] MFA Rules not processed for user `815705d191904e6f977b68273f59b66a`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:05.176812 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-d29f91d5-7474-4010-b2be-7766065f12cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:05.245952 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1081/2169] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:05.260863 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61722677-ad3e-45c2-8230-74ab90623384 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:05.262786 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-61722677-ad3e-45c2-8230-74ab90623384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:05.329402 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61722677-ad3e-45c2-8230-74ab90623384 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:05.335317 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64438ce-094d-418a-8312-a1da7740d949 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:05.338023 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64438ce-094d-418a-8312-a1da7740d949 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:05.338317 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64438ce-094d-418a-8312-a1da7740d949 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:05.338597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64438ce-094d-418a-8312-a1da7740d949 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:05.351582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64438ce-094d-418a-8312-a1da7740d949 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:05.353233 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64438ce-094d-418a-8312-a1da7740d949 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:05.363144 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1089/2170] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:02:05.549811 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f91d5-7474-4010-b2be-7766065f12cf None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:05.551755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-d29f91d5-7474-4010-b2be-7766065f12cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:05.564471 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f91d5-7474-4010-b2be-7766065f12cf None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:05.568467 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4887ab-6b28-40cf-88ec-3f27f332e403 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:05.569469 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4887ab-6b28-40cf-88ec-3f27f332e403 service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:05.569718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4887ab-6b28-40cf-88ec-3f27f332e403 service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:05.570202 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4887ab-6b28-40cf-88ec-3f27f332e403 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:05.572169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-3a4887ab-6b28-40cf-88ec-3f27f332e403 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:05.647227 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4887ab-6b28-40cf-88ec-3f27f332e403 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:05.649199 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4887ab-6b28-40cf-88ec-3f27f332e403 service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:05.659506 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1082/2171] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:02:08.488317 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64438ce-094d-418a-8312-a1da7740d949 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:08.497917 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64438ce-094d-418a-8312-a1da7740d949 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:08.504577 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea36601-2ba4-413e-9d9f-2d75fa304f7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:08.505252 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea36601-2ba4-413e-9d9f-2d75fa304f7b service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:08.505562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea36601-2ba4-413e-9d9f-2d75fa304f7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:08.505942 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea36601-2ba4-413e-9d9f-2d75fa304f7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:08.507950 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fea36601-2ba4-413e-9d9f-2d75fa304f7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:08.548140 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:08.581366 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea36601-2ba4-413e-9d9f-2d75fa304f7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:08.582785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea36601-2ba4-413e-9d9f-2d75fa304f7b service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:08.593404 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1090/2172] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:08 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:02:09.739258 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4887ab-6b28-40cf-88ec-3f27f332e403 service cinder] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:09.742422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-3a4887ab-6b28-40cf-88ec-3f27f332e403 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:09.805426 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:09.833002 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4887ab-6b28-40cf-88ec-3f27f332e403 service cinder] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:09.837984 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f21e94-7cd8-4fe2-a39a-8d0f0eaab910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:09.838877 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f21e94-7cd8-4fe2-a39a-8d0f0eaab910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:09.839003 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f21e94-7cd8-4fe2-a39a-8d0f0eaab910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:09.839166 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f21e94-7cd8-4fe2-a39a-8d0f0eaab910 admin admin] PATH_INFO: `/v3/users/857f89085c794bd1b746f74675bd246f` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:09.840564 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f21e94-7cd8-4fe2-a39a-8d0f0eaab910 admin admin] RBAC: Authorizing `identity:delete_user(user_id=857f89085c794bd1b746f74675bd246f)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:09.843185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f21e94-7cd8-4fe2-a39a-8d0f0eaab910 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:09.898383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c3f21e94-7cd8-4fe2-a39a-8d0f0eaab910 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '857f89085c794bd1b746f74675bd246f'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:09.906346 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c3f21e94-7cd8-4fe2-a39a-8d0f0eaab910 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '857f89085c794bd1b746f74675bd246f'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:09.917873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c3f21e94-7cd8-4fe2-a39a-8d0f0eaab910 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '857f89085c794bd1b746f74675bd246f'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:09.918422 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c3f21e94-7cd8-4fe2-a39a-8d0f0eaab910 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '857f89085c794bd1b746f74675bd246f'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:09.919199 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c3f21e94-7cd8-4fe2-a39a-8d0f0eaab910 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '857f89085c794bd1b746f74675bd246f'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:09.944497 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1083/2173] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:02:09 2022] DELETE /identity/v3/users/857f89085c794bd1b746f74675bd246f => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:09.949769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d26dfa-9eb9-4323-aa6e-b006301ac260 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:09.950272 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d26dfa-9eb9-4323-aa6e-b006301ac260 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:09.950362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d26dfa-9eb9-4323-aa6e-b006301ac260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:09.969954 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-77d26dfa-9eb9-4323-aa6e-b006301ac260 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:09.971507 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-77d26dfa-9eb9-4323-aa6e-b006301ac260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:10.066455 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1091/2174] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:10.072895 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f21e94-7cd8-4fe2-a39a-8d0f0eaab910 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:10.074701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c3f21e94-7cd8-4fe2-a39a-8d0f0eaab910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:10.136923 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f21e94-7cd8-4fe2-a39a-8d0f0eaab910 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:10.140995 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012bfc43-c143-4d2b-a31b-c0f05e553e17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:10.141654 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012bfc43-c143-4d2b-a31b-c0f05e553e17 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:10.141936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012bfc43-c143-4d2b-a31b-c0f05e553e17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:10.142264 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012bfc43-c143-4d2b-a31b-c0f05e553e17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:10.157809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012bfc43-c143-4d2b-a31b-c0f05e553e17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:10.157809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012bfc43-c143-4d2b-a31b-c0f05e553e17 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:10.165275 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1084/2175] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:02:10.464962 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d26dfa-9eb9-4323-aa6e-b006301ac260 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:10.466406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-77d26dfa-9eb9-4323-aa6e-b006301ac260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:10.487623 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d26dfa-9eb9-4323-aa6e-b006301ac260 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:10.492973 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57138d1-a399-4d17-87ca-67e827f8e30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:10.493895 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57138d1-a399-4d17-87ca-67e827f8e30f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:10.494340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57138d1-a399-4d17-87ca-67e827f8e30f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:10.494864 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57138d1-a399-4d17-87ca-67e827f8e30f admin admin] PATH_INFO: `/v3/projects/b43e812e5c254c13be16ba66a616a79e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:10.496373 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57138d1-a399-4d17-87ca-67e827f8e30f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b43e812e5c254c13be16ba66a616a79e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:10.498406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57138d1-a399-4d17-87ca-67e827f8e30f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:10.549611 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c57138d1-a399-4d17-87ca-67e827f8e30f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b43e812e5c254c13be16ba66a616a79e', 'request_id': 'req-c57138d1-a399-4d17-87ca-67e827f8e30f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:10.556246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c57138d1-a399-4d17-87ca-67e827f8e30f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b43e812e5c254c13be16ba66a616a79e', 'request_id': 'req-c57138d1-a399-4d17-87ca-67e827f8e30f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:10.564691 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c57138d1-a399-4d17-87ca-67e827f8e30f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b43e812e5c254c13be16ba66a616a79e', 'request_id': 'req-c57138d1-a399-4d17-87ca-67e827f8e30f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:10.576915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c57138d1-a399-4d17-87ca-67e827f8e30f admin admin] The token cache is being invalidate because project b43e812e5c254c13be16ba66a616a79e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:10.577343 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c57138d1-a399-4d17-87ca-67e827f8e30f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:10.578226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c57138d1-a399-4d17-87ca-67e827f8e30f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:10.580057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1092/2176] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:10 2022] DELETE /identity/v3/projects/b43e812e5c254c13be16ba66a616a79e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:10.584438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012bfc43-c143-4d2b-a31b-c0f05e553e17 service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:10.595424 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012bfc43-c143-4d2b-a31b-c0f05e553e17 service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:10.657104 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd8b7f-e2ae-44b8-9275-2ef4e746cc1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:10.657769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd8b7f-e2ae-44b8-9275-2ef4e746cc1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:10.658052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd8b7f-e2ae-44b8-9275-2ef4e746cc1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:10.658327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd8b7f-e2ae-44b8-9275-2ef4e746cc1f admin admin] PATH_INFO: `/v3/users/cc3ed6a7330f44e18fd480d317fd243a` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:10.659565 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd8b7f-e2ae-44b8-9275-2ef4e746cc1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc3ed6a7330f44e18fd480d317fd243a)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:10.661562 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd8b7f-e2ae-44b8-9275-2ef4e746cc1f admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:10.663194 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:10.731278 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-a2bd8b7f-e2ae-44b8-9275-2ef4e746cc1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc3ed6a7330f44e18fd480d317fd243a'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:10.739361 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-a2bd8b7f-e2ae-44b8-9275-2ef4e746cc1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc3ed6a7330f44e18fd480d317fd243a'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:10.748440 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-a2bd8b7f-e2ae-44b8-9275-2ef4e746cc1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc3ed6a7330f44e18fd480d317fd243a'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:10.749137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-a2bd8b7f-e2ae-44b8-9275-2ef4e746cc1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc3ed6a7330f44e18fd480d317fd243a'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:10.749804 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-a2bd8b7f-e2ae-44b8-9275-2ef4e746cc1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc3ed6a7330f44e18fd480d317fd243a'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:10.771635 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1085/2177] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:02:10 2022] DELETE /identity/v3/users/cc3ed6a7330f44e18fd480d317fd243a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:11.153958 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57138d1-a399-4d17-87ca-67e827f8e30f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:11.155623 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c57138d1-a399-4d17-87ca-67e827f8e30f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:11.169326 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57138d1-a399-4d17-87ca-67e827f8e30f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:11.220421 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:11.241152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34520f4c-a39b-47ba-bf6b-5cdfa036a38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:11.242007 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34520f4c-a39b-47ba-bf6b-5cdfa036a38d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:11.242442 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34520f4c-a39b-47ba-bf6b-5cdfa036a38d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:11.242874 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34520f4c-a39b-47ba-bf6b-5cdfa036a38d admin admin] PATH_INFO: `/v3/projects/4918431a1cc34242a77903d241406bf4` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:11.244272 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34520f4c-a39b-47ba-bf6b-5cdfa036a38d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4918431a1cc34242a77903d241406bf4)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:11.246836 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34520f4c-a39b-47ba-bf6b-5cdfa036a38d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:11.318700 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-34520f4c-a39b-47ba-bf6b-5cdfa036a38d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4918431a1cc34242a77903d241406bf4', 'request_id': 'req-34520f4c-a39b-47ba-bf6b-5cdfa036a38d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:11.327220 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-34520f4c-a39b-47ba-bf6b-5cdfa036a38d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4918431a1cc34242a77903d241406bf4', 'request_id': 'req-34520f4c-a39b-47ba-bf6b-5cdfa036a38d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:11.344380 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-34520f4c-a39b-47ba-bf6b-5cdfa036a38d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4918431a1cc34242a77903d241406bf4', 'request_id': 'req-34520f4c-a39b-47ba-bf6b-5cdfa036a38d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:11.357970 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-34520f4c-a39b-47ba-bf6b-5cdfa036a38d admin admin] The token cache is being invalidate because project 4918431a1cc34242a77903d241406bf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:11.358257 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-34520f4c-a39b-47ba-bf6b-5cdfa036a38d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:11.359071 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-34520f4c-a39b-47ba-bf6b-5cdfa036a38d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:11.360845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1093/2178] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:11 2022] DELETE /identity/v3/projects/4918431a1cc34242a77903d241406bf4 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:11.365488 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f61cc-007f-4d69-9155-029e610eae1e None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:11.365787 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f61cc-007f-4d69-9155-029e610eae1e None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:11.366057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f61cc-007f-4d69-9155-029e610eae1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:11.386423 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-5a9f61cc-007f-4d69-9155-029e610eae1e None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:11.387745 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-5a9f61cc-007f-4d69-9155-029e610eae1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:11.479501 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1086/2179] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:11.483394 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba2b1c-a82b-4048-8d98-7d36bc0dd0a2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:11.483881 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba2b1c-a82b-4048-8d98-7d36bc0dd0a2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:11.484320 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba2b1c-a82b-4048-8d98-7d36bc0dd0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:11.512605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-65ba2b1c-a82b-4048-8d98-7d36bc0dd0a2 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:11.514342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-65ba2b1c-a82b-4048-8d98-7d36bc0dd0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:11.527567 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1094/2180] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:11.532057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f61cc-007f-4d69-9155-029e610eae1e None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:11.542133 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f61cc-007f-4d69-9155-029e610eae1e None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:11.547156 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49fa216-f4f6-44fd-9eed-f088e322dd02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:11.547816 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fa216-f4f6-44fd-9eed-f088e322dd02 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:11.548101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fa216-f4f6-44fd-9eed-f088e322dd02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:11.548379 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fa216-f4f6-44fd-9eed-f088e322dd02 admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:11.549087 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49fa216-f4f6-44fd-9eed-f088e322dd02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:02:11.549395 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49fa216-f4f6-44fd-9eed-f088e322dd02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:11.551023 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49fa216-f4f6-44fd-9eed-f088e322dd02 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:11.562647 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1087/2181] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:02:11.567580 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba2b1c-a82b-4048-8d98-7d36bc0dd0a2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:11.577470 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba2b1c-a82b-4048-8d98-7d36bc0dd0a2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:11.582253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be4917-9ada-4c83-9753-79f390494185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:11.583081 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be4917-9ada-4c83-9753-79f390494185 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:11.583361 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be4917-9ada-4c83-9753-79f390494185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:11.583666 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be4917-9ada-4c83-9753-79f390494185 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:11.584352 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be4917-9ada-4c83-9753-79f390494185 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:11.586296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be4917-9ada-4c83-9753-79f390494185 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:11.615058 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1095/2182] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:02:11 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:02:11.620788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49fa216-f4f6-44fd-9eed-f088e322dd02 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:11.632055 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49fa216-f4f6-44fd-9eed-f088e322dd02 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:11.692226 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81549087-cb78-4d43-8ced-372065613446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:11.692988 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81549087-cb78-4d43-8ced-372065613446 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:11.693418 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81549087-cb78-4d43-8ced-372065613446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:11.693859 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81549087-cb78-4d43-8ced-372065613446 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:11.694883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81549087-cb78-4d43-8ced-372065613446 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:11.696935 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81549087-cb78-4d43-8ced-372065613446 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:11.710002 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:11.723458 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1088/2183] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:02:11 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:11.728077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be4917-9ada-4c83-9753-79f390494185 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:11.739761 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be4917-9ada-4c83-9753-79f390494185 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:11.745992 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa0e2b-b97d-47d6-bc6a-887e84556620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:11.746715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa0e2b-b97d-47d6-bc6a-887e84556620 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:11.747066 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa0e2b-b97d-47d6-bc6a-887e84556620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:11.747419 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa0e2b-b97d-47d6-bc6a-887e84556620 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:11.748228 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa0e2b-b97d-47d6-bc6a-887e84556620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:11.749899 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa0e2b-b97d-47d6-bc6a-887e84556620 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:11.765780 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1096/2184] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:11.768775 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81549087-cb78-4d43-8ced-372065613446 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:11.779672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81549087-cb78-4d43-8ced-372065613446 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:11.785059 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbe083-153a-4ef0-906c-dfd675c2addc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:11.785704 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbe083-153a-4ef0-906c-dfd675c2addc admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:11.785987 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbe083-153a-4ef0-906c-dfd675c2addc admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:11.786259 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbe083-153a-4ef0-906c-dfd675c2addc admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:11.786969 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cbe083-153a-4ef0-906c-dfd675c2addc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:11.788636 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cbe083-153a-4ef0-906c-dfd675c2addc admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:11.806584 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1089/2185] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:11.811635 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa0e2b-b97d-47d6-bc6a-887e84556620 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:11.823641 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa0e2b-b97d-47d6-bc6a-887e84556620 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:11.828791 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc35d9-c9ee-4992-bc02-f2a24de3b8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:11.829547 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc35d9-c9ee-4992-bc02-f2a24de3b8e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:11.829883 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc35d9-c9ee-4992-bc02-f2a24de3b8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:11.830167 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc35d9-c9ee-4992-bc02-f2a24de3b8e2 admin admin] PATH_INFO: `/v3/projects/df3032328e6c4113b585e8ca7bde3dfe/users/d70205bb06aa4a4e96efa201b33bd631/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:11.845727 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc35d9-c9ee-4992-bc02-f2a24de3b8e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df3032328e6c4113b585e8ca7bde3dfe, user_id=d70205bb06aa4a4e96efa201b33bd631, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:11.848408 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc35d9-c9ee-4992-bc02-f2a24de3b8e2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:11.863102 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1097/2186] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:11 2022] PUT /identity/v3/projects/df3032328e6c4113b585e8ca7bde3dfe/users/d70205bb06aa4a4e96efa201b33bd631/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:11.876492 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18fa4a-0645-4306-b8d2-cc52ff68912a None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:11.876790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18fa4a-0645-4306-b8d2-cc52ff68912a None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:11.877292 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18fa4a-0645-4306-b8d2-cc52ff68912a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:11.909274 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-bc18fa4a-0645-4306-b8d2-cc52ff68912a None None] MFA Rules not processed for user `d70205bb06aa4a4e96efa201b33bd631`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:11.910561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-bc18fa4a-0645-4306-b8d2-cc52ff68912a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:11.977591 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1090/2187] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:11.982189 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc35d9-c9ee-4992-bc02-f2a24de3b8e2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:12.074237 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc35d9-c9ee-4992-bc02-f2a24de3b8e2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:12.079472 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc8f04-fc72-4736-807b-3aa67723c274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:12.080330 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc8f04-fc72-4736-807b-3aa67723c274 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:12.080617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc8f04-fc72-4736-807b-3aa67723c274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:12.081708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc8f04-fc72-4736-807b-3aa67723c274 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:12.081708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc8f04-fc72-4736-807b-3aa67723c274 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:12.083492 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc8f04-fc72-4736-807b-3aa67723c274 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:12.117239 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1098/2188] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:02:11 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:12.121205 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18fa4a-0645-4306-b8d2-cc52ff68912a None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:12.137724 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18fa4a-0645-4306-b8d2-cc52ff68912a None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:12.200133 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3cab22-2c17-41fb-8fbc-08684b3df557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:12.200684 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3cab22-2c17-41fb-8fbc-08684b3df557 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:12.200972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3cab22-2c17-41fb-8fbc-08684b3df557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:12.201247 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3cab22-2c17-41fb-8fbc-08684b3df557 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:12.201925 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3cab22-2c17-41fb-8fbc-08684b3df557 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:12.203950 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3cab22-2c17-41fb-8fbc-08684b3df557 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:12.232586 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1091/2189] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:02:12 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:02:12.236597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc8f04-fc72-4736-807b-3aa67723c274 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:12.242683 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:12.249226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc8f04-fc72-4736-807b-3aa67723c274 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:12.255269 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed18ef4-308f-488c-b562-d953b212e470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:12.255933 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed18ef4-308f-488c-b562-d953b212e470 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:12.256214 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed18ef4-308f-488c-b562-d953b212e470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:12.256486 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed18ef4-308f-488c-b562-d953b212e470 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:12.257283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed18ef4-308f-488c-b562-d953b212e470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:12.258912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed18ef4-308f-488c-b562-d953b212e470 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:12.269756 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1099/2190] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:12.274403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3cab22-2c17-41fb-8fbc-08684b3df557 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:12.285272 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3cab22-2c17-41fb-8fbc-08684b3df557 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:12.290757 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d96bde-7bdc-4903-a060-62862ef3e307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:12.291507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d96bde-7bdc-4903-a060-62862ef3e307 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:12.291880 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d96bde-7bdc-4903-a060-62862ef3e307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:12.292214 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d96bde-7bdc-4903-a060-62862ef3e307 admin admin] PATH_INFO: `/v3/projects/f025fef40678400ca3b025db20fe3f30/users/bd141749fff046dc85f689fdf770d129/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:12.309054 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d96bde-7bdc-4903-a060-62862ef3e307 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f025fef40678400ca3b025db20fe3f30, user_id=bd141749fff046dc85f689fdf770d129, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:12.312098 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d96bde-7bdc-4903-a060-62862ef3e307 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:12.326725 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1092/2191] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:12 2022] PUT /identity/v3/projects/f025fef40678400ca3b025db20fe3f30/users/bd141749fff046dc85f689fdf770d129/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:12.330989 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed18ef4-308f-488c-b562-d953b212e470 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:12.343182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed18ef4-308f-488c-b562-d953b212e470 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:12.401062 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8063185b-3324-4d8a-81fc-6fa25e8ab640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:12.401696 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063185b-3324-4d8a-81fc-6fa25e8ab640 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:12.401970 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063185b-3324-4d8a-81fc-6fa25e8ab640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:12.402252 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063185b-3324-4d8a-81fc-6fa25e8ab640 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:12.402947 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8063185b-3324-4d8a-81fc-6fa25e8ab640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:12.404571 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8063185b-3324-4d8a-81fc-6fa25e8ab640 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:12.417911 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1100/2192] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:12.421383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d96bde-7bdc-4903-a060-62862ef3e307 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:12.432528 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d96bde-7bdc-4903-a060-62862ef3e307 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:12.437586 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c6478-e40c-4e80-86ff-cda893c9be7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:12.438278 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c6478-e40c-4e80-86ff-cda893c9be7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:12.438561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c6478-e40c-4e80-86ff-cda893c9be7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:12.438839 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c6478-e40c-4e80-86ff-cda893c9be7f admin admin] PATH_INFO: `/v3/projects/f025fef40678400ca3b025db20fe3f30/users/bd141749fff046dc85f689fdf770d129/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:12.441125 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c6478-e40c-4e80-86ff-cda893c9be7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f025fef40678400ca3b025db20fe3f30, user_id=bd141749fff046dc85f689fdf770d129, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:12.444117 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c6478-e40c-4e80-86ff-cda893c9be7f admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:12.456368 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1093/2193] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:12 2022] PUT /identity/v3/projects/f025fef40678400ca3b025db20fe3f30/users/bd141749fff046dc85f689fdf770d129/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:02:12.466809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358445c-7844-41f3-a7ef-15ac2d33691c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:12.467093 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358445c-7844-41f3-a7ef-15ac2d33691c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:12.467514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358445c-7844-41f3-a7ef-15ac2d33691c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:12.509141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b358445c-7844-41f3-a7ef-15ac2d33691c None None] MFA Rules not processed for user `bd141749fff046dc85f689fdf770d129`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:12.509141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b358445c-7844-41f3-a7ef-15ac2d33691c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:12.597122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1101/2194] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:12.598796 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c6478-e40c-4e80-86ff-cda893c9be7f admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:12.603995 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-5f3c6478-e40c-4e80-86ff-cda893c9be7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:12.610408 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b358445c-7844-41f3-a7ef-15ac2d33691c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:12.611925 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b358445c-7844-41f3-a7ef-15ac2d33691c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:12.641818 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:12.671379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b358445c-7844-41f3-a7ef-15ac2d33691c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:12.675381 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21347548-9805-4792-a087-e5c88e2b73f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:12.676041 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21347548-9805-4792-a087-e5c88e2b73f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:12.676337 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21347548-9805-4792-a087-e5c88e2b73f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:12.676611 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21347548-9805-4792-a087-e5c88e2b73f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:12.683232 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c6478-e40c-4e80-86ff-cda893c9be7f admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:12.687455 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21347548-9805-4792-a087-e5c88e2b73f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:12.688490 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8319a-53f1-45bc-a689-f669d3024c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:12.689153 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8319a-53f1-45bc-a689-f669d3024c7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:12.689239 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21347548-9805-4792-a087-e5c88e2b73f6 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:12.689456 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8319a-53f1-45bc-a689-f669d3024c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:12.689764 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8319a-53f1-45bc-a689-f669d3024c7b admin admin] PATH_INFO: `/v3/users/c3c8f928717c4a03816d70bbee64d1c0` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:12.690957 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8319a-53f1-45bc-a689-f669d3024c7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3c8f928717c4a03816d70bbee64d1c0)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:12.692946 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8319a-53f1-45bc-a689-f669d3024c7b admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:12.701042 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1102/2195] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:12 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:02:12.755621 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-dda8319a-53f1-45bc-a689-f669d3024c7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3c8f928717c4a03816d70bbee64d1c0'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:12.764949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-dda8319a-53f1-45bc-a689-f669d3024c7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3c8f928717c4a03816d70bbee64d1c0'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:12.774133 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-dda8319a-53f1-45bc-a689-f669d3024c7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3c8f928717c4a03816d70bbee64d1c0'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:12.775577 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-dda8319a-53f1-45bc-a689-f669d3024c7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3c8f928717c4a03816d70bbee64d1c0'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:12.776360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-dda8319a-53f1-45bc-a689-f669d3024c7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3c8f928717c4a03816d70bbee64d1c0'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:12.799631 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1094/2196] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:02:12 2022] DELETE /identity/v3/users/c3c8f928717c4a03816d70bbee64d1c0 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:02:12.806273 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbf997-300d-4bc3-bcbd-d2cf484abd5c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:12.807049 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbf997-300d-4bc3-bcbd-d2cf484abd5c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:12.807676 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbf997-300d-4bc3-bcbd-d2cf484abd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:12.833382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-dffbf997-300d-4bc3-bcbd-d2cf484abd5c None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:12.834772 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dffbf997-300d-4bc3-bcbd-d2cf484abd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:12.846708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:12.924388 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1103/2197] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:12.934491 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8319a-53f1-45bc-a689-f669d3024c7b admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:12.936174 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-dda8319a-53f1-45bc-a689-f669d3024c7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:13.000339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8319a-53f1-45bc-a689-f669d3024c7b admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:13.005964 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56690a4a-c281-451a-a9bb-146b883a519b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:13.006686 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56690a4a-c281-451a-a9bb-146b883a519b service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:13.006938 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56690a4a-c281-451a-a9bb-146b883a519b service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:13.007138 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56690a4a-c281-451a-a9bb-146b883a519b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:13.018173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56690a4a-c281-451a-a9bb-146b883a519b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:13.019694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56690a4a-c281-451a-a9bb-146b883a519b service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:13.030249 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1095/2198] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:02:13.408371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffbf997-300d-4bc3-bcbd-d2cf484abd5c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:13.408371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dffbf997-300d-4bc3-bcbd-d2cf484abd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:13.413503 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56690a4a-c281-451a-a9bb-146b883a519b service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:13.414234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-56690a4a-c281-451a-a9bb-146b883a519b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:13.418717 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffbf997-300d-4bc3-bcbd-d2cf484abd5c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:13.436853 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d207b93-223c-4c44-8953-5c9ab92ce121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:13.437533 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d207b93-223c-4c44-8953-5c9ab92ce121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:13.437817 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d207b93-223c-4c44-8953-5c9ab92ce121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:13.438092 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d207b93-223c-4c44-8953-5c9ab92ce121 admin admin] PATH_INFO: `/v3/projects/7b90b49d3f014f7baed569a0f9a1d1fa` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:13.446274 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56690a4a-c281-451a-a9bb-146b883a519b service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:13.449096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d207b93-223c-4c44-8953-5c9ab92ce121 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b90b49d3f014f7baed569a0f9a1d1fa)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:13.451353 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb7e68-24a5-4471-a916-d1487c65d54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:13.452037 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb7e68-24a5-4471-a916-d1487c65d54a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:13.452440 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d207b93-223c-4c44-8953-5c9ab92ce121 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:13.452547 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb7e68-24a5-4471-a916-d1487c65d54a admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:13.452730 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb7e68-24a5-4471-a916-d1487c65d54a admin admin] PATH_INFO: `/v3/users/d70205bb06aa4a4e96efa201b33bd631` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:13.453888 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb7e68-24a5-4471-a916-d1487c65d54a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d70205bb06aa4a4e96efa201b33bd631)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:13.455867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb7e68-24a5-4471-a916-d1487c65d54a admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:13.515804 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4d207b93-223c-4c44-8953-5c9ab92ce121 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b90b49d3f014f7baed569a0f9a1d1fa', 'request_id': 'req-4d207b93-223c-4c44-8953-5c9ab92ce121'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:13.523035 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:13.525245 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4d207b93-223c-4c44-8953-5c9ab92ce121 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b90b49d3f014f7baed569a0f9a1d1fa', 'request_id': 'req-4d207b93-223c-4c44-8953-5c9ab92ce121'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:13.527334 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-39fb7e68-24a5-4471-a916-d1487c65d54a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd70205bb06aa4a4e96efa201b33bd631'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:13.536446 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4d207b93-223c-4c44-8953-5c9ab92ce121 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b90b49d3f014f7baed569a0f9a1d1fa', 'request_id': 'req-4d207b93-223c-4c44-8953-5c9ab92ce121'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:13.547239 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-39fb7e68-24a5-4471-a916-d1487c65d54a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd70205bb06aa4a4e96efa201b33bd631'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:13.552611 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4d207b93-223c-4c44-8953-5c9ab92ce121 admin admin] The token cache is being invalidate because project 7b90b49d3f014f7baed569a0f9a1d1fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:13.552994 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4d207b93-223c-4c44-8953-5c9ab92ce121 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:13.553835 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4d207b93-223c-4c44-8953-5c9ab92ce121 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:13.555959 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1104/2199] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:13 2022] DELETE /identity/v3/projects/7b90b49d3f014f7baed569a0f9a1d1fa => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:13.562724 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d207b93-223c-4c44-8953-5c9ab92ce121 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:13.563243 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-39fb7e68-24a5-4471-a916-d1487c65d54a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd70205bb06aa4a4e96efa201b33bd631'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:13.564054 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-39fb7e68-24a5-4471-a916-d1487c65d54a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd70205bb06aa4a4e96efa201b33bd631'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:13.564435 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4d207b93-223c-4c44-8953-5c9ab92ce121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:13.567095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-39fb7e68-24a5-4471-a916-d1487c65d54a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd70205bb06aa4a4e96efa201b33bd631'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:13.594684 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1096/2200] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:02:13 2022] DELETE /identity/v3/users/d70205bb06aa4a4e96efa201b33bd631 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:02:13.600600 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ad563-d511-4e0a-a764-a7ee5887aa4f None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:13.601098 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ad563-d511-4e0a-a764-a7ee5887aa4f None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:13.601511 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ad563-d511-4e0a-a764-a7ee5887aa4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:13.638820 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-bb0ad563-d511-4e0a-a764-a7ee5887aa4f None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:13.640165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-bb0ad563-d511-4e0a-a764-a7ee5887aa4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:13.669446 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d207b93-223c-4c44-8953-5c9ab92ce121 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:13.717808 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1097/2201] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:13.725484 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:13.727645 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ad563-d511-4e0a-a764-a7ee5887aa4f None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:13.729410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-bb0ad563-d511-4e0a-a764-a7ee5887aa4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:13.743872 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105b34d-d0aa-4942-a0b9-806b7d40693e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:13.744476 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105b34d-d0aa-4942-a0b9-806b7d40693e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:13.747423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105b34d-d0aa-4942-a0b9-806b7d40693e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:13.747814 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105b34d-d0aa-4942-a0b9-806b7d40693e admin admin] PATH_INFO: `/v3/users/57e500a5809547d39b1c0d7342d37920` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:13.749175 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3105b34d-d0aa-4942-a0b9-806b7d40693e admin admin] RBAC: Authorizing `identity:delete_user(user_id=57e500a5809547d39b1c0d7342d37920)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:13.751250 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3105b34d-d0aa-4942-a0b9-806b7d40693e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:13.798228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ad563-d511-4e0a-a764-a7ee5887aa4f None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:13.802203 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbaa7a3-2be5-4894-966f-61b99e0d42b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:13.802925 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbaa7a3-2be5-4894-966f-61b99e0d42b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:13.803314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbaa7a3-2be5-4894-966f-61b99e0d42b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:13.803694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbaa7a3-2be5-4894-966f-61b99e0d42b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:13.806846 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3105b34d-d0aa-4942-a0b9-806b7d40693e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57e500a5809547d39b1c0d7342d37920'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:13.814179 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbaa7a3-2be5-4894-966f-61b99e0d42b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:13.815126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3105b34d-d0aa-4942-a0b9-806b7d40693e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57e500a5809547d39b1c0d7342d37920'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:13.815839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3105b34d-d0aa-4942-a0b9-806b7d40693e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57e500a5809547d39b1c0d7342d37920'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:13.816036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbaa7a3-2be5-4894-966f-61b99e0d42b0 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:13.825274 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3105b34d-d0aa-4942-a0b9-806b7d40693e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57e500a5809547d39b1c0d7342d37920'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:13.825936 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3105b34d-d0aa-4942-a0b9-806b7d40693e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57e500a5809547d39b1c0d7342d37920'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:13.828845 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1098/2202] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:02:13.852405 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1105/2203] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:02:13 2022] DELETE /identity/v3/users/57e500a5809547d39b1c0d7342d37920 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:14.263980 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbaa7a3-2be5-4894-966f-61b99e0d42b0 service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:14.266033 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-8fbaa7a3-2be5-4894-966f-61b99e0d42b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:14.281176 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbaa7a3-2be5-4894-966f-61b99e0d42b0 service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:14.284060 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105b34d-d0aa-4942-a0b9-806b7d40693e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:14.285620 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3105b34d-d0aa-4942-a0b9-806b7d40693e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:14.358078 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec924e-3657-46c8-b164-d4b8ef2a1840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:14.359478 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec924e-3657-46c8-b164-d4b8ef2a1840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:14.359963 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec924e-3657-46c8-b164-d4b8ef2a1840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:14.360358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec924e-3657-46c8-b164-d4b8ef2a1840 admin admin] PATH_INFO: `/v3/projects/df3032328e6c4113b585e8ca7bde3dfe` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:14.361741 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec924e-3657-46c8-b164-d4b8ef2a1840 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df3032328e6c4113b585e8ca7bde3dfe)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:14.364035 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec924e-3657-46c8-b164-d4b8ef2a1840 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:14.372457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:14.372617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105b34d-d0aa-4942-a0b9-806b7d40693e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:14.378199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb98468-2cbd-42ef-a70b-add290c2e482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:14.379261 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb98468-2cbd-42ef-a70b-add290c2e482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:14.379913 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb98468-2cbd-42ef-a70b-add290c2e482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:14.379913 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb98468-2cbd-42ef-a70b-add290c2e482 admin admin] PATH_INFO: `/v3/projects/95f9eae1576f47ebb786c92a32f7fa24` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:14.381082 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb98468-2cbd-42ef-a70b-add290c2e482 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95f9eae1576f47ebb786c92a32f7fa24)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:14.383163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb98468-2cbd-42ef-a70b-add290c2e482 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:14.420469 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c9ec924e-3657-46c8-b164-d4b8ef2a1840 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df3032328e6c4113b585e8ca7bde3dfe', 'request_id': 'req-c9ec924e-3657-46c8-b164-d4b8ef2a1840'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:14.428967 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c9ec924e-3657-46c8-b164-d4b8ef2a1840 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df3032328e6c4113b585e8ca7bde3dfe', 'request_id': 'req-c9ec924e-3657-46c8-b164-d4b8ef2a1840'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:14.437271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c9ec924e-3657-46c8-b164-d4b8ef2a1840 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df3032328e6c4113b585e8ca7bde3dfe', 'request_id': 'req-c9ec924e-3657-46c8-b164-d4b8ef2a1840'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:14.446551 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aeb98468-2cbd-42ef-a70b-add290c2e482 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95f9eae1576f47ebb786c92a32f7fa24', 'request_id': 'req-aeb98468-2cbd-42ef-a70b-add290c2e482'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:14.448987 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c9ec924e-3657-46c8-b164-d4b8ef2a1840 admin admin] The token cache is being invalidate because project df3032328e6c4113b585e8ca7bde3dfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:14.449349 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c9ec924e-3657-46c8-b164-d4b8ef2a1840 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:14.450238 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c9ec924e-3657-46c8-b164-d4b8ef2a1840 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:14.452139 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1099/2204] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:14 2022] DELETE /identity/v3/projects/df3032328e6c4113b585e8ca7bde3dfe => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:14.452967 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec924e-3657-46c8-b164-d4b8ef2a1840 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:14.456293 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c9ec924e-3657-46c8-b164-d4b8ef2a1840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:14.457051 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aeb98468-2cbd-42ef-a70b-add290c2e482 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95f9eae1576f47ebb786c92a32f7fa24', 'request_id': 'req-aeb98468-2cbd-42ef-a70b-add290c2e482'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:14.468456 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aeb98468-2cbd-42ef-a70b-add290c2e482 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95f9eae1576f47ebb786c92a32f7fa24', 'request_id': 'req-aeb98468-2cbd-42ef-a70b-add290c2e482'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:14.476142 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:14.483301 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aeb98468-2cbd-42ef-a70b-add290c2e482 admin admin] The token cache is being invalidate because project 95f9eae1576f47ebb786c92a32f7fa24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:14.483761 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aeb98468-2cbd-42ef-a70b-add290c2e482 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:14.485025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aeb98468-2cbd-42ef-a70b-add290c2e482 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:14.487225 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1106/2205] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:14 2022] DELETE /identity/v3/projects/95f9eae1576f47ebb786c92a32f7fa24 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:02:14.488019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb98468-2cbd-42ef-a70b-add290c2e482 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:14.492338 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-aeb98468-2cbd-42ef-a70b-add290c2e482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:14.557279 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec924e-3657-46c8-b164-d4b8ef2a1840 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:14.621001 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb98468-2cbd-42ef-a70b-add290c2e482 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:14.643993 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918a2f1-d8e5-4cc5-9fc1-68339297695b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:14.644927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918a2f1-d8e5-4cc5-9fc1-68339297695b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:14.645516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918a2f1-d8e5-4cc5-9fc1-68339297695b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:14.645813 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918a2f1-d8e5-4cc5-9fc1-68339297695b admin admin] PATH_INFO: `/v3/users/bd141749fff046dc85f689fdf770d129` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:14.647688 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c918a2f1-d8e5-4cc5-9fc1-68339297695b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd141749fff046dc85f689fdf770d129)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:14.656848 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c918a2f1-d8e5-4cc5-9fc1-68339297695b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:14.670326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227186f5-ee88-48fd-87b8-1b24575b4f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:14.671188 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227186f5-ee88-48fd-87b8-1b24575b4f2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:14.671488 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227186f5-ee88-48fd-87b8-1b24575b4f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:14.671870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227186f5-ee88-48fd-87b8-1b24575b4f2d admin admin] PATH_INFO: `/v3/users/08a8b9fcf94a4be39ec5468bc24c6dfd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:14.673077 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227186f5-ee88-48fd-87b8-1b24575b4f2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=08a8b9fcf94a4be39ec5468bc24c6dfd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:14.675060 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227186f5-ee88-48fd-87b8-1b24575b4f2d admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:14.720225 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c918a2f1-d8e5-4cc5-9fc1-68339297695b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd141749fff046dc85f689fdf770d129'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:14.720866 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:14.723736 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-227186f5-ee88-48fd-87b8-1b24575b4f2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08a8b9fcf94a4be39ec5468bc24c6dfd'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:14.729725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c918a2f1-d8e5-4cc5-9fc1-68339297695b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd141749fff046dc85f689fdf770d129'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:14.731205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c918a2f1-d8e5-4cc5-9fc1-68339297695b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd141749fff046dc85f689fdf770d129'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:14.733115 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-227186f5-ee88-48fd-87b8-1b24575b4f2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08a8b9fcf94a4be39ec5468bc24c6dfd'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:14.739862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c918a2f1-d8e5-4cc5-9fc1-68339297695b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd141749fff046dc85f689fdf770d129'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:14.740602 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c918a2f1-d8e5-4cc5-9fc1-68339297695b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd141749fff046dc85f689fdf770d129'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:14.742846 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-227186f5-ee88-48fd-87b8-1b24575b4f2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08a8b9fcf94a4be39ec5468bc24c6dfd'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:14.743571 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-227186f5-ee88-48fd-87b8-1b24575b4f2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08a8b9fcf94a4be39ec5468bc24c6dfd'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:14.744306 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-227186f5-ee88-48fd-87b8-1b24575b4f2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08a8b9fcf94a4be39ec5468bc24c6dfd'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:14.760018 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1107/2206] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:02:14 2022] DELETE /identity/v3/users/bd141749fff046dc85f689fdf770d129 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:14.761185 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918a2f1-d8e5-4cc5-9fc1-68339297695b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:14.762261 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1100/2207] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:02:14 2022] DELETE /identity/v3/users/08a8b9fcf94a4be39ec5468bc24c6dfd => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:14.763257 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c918a2f1-d8e5-4cc5-9fc1-68339297695b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:14.766501 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf17ea-5576-434d-b2b5-26ab9b1eeeea None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:14.766852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf17ea-5576-434d-b2b5-26ab9b1eeeea None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:14.767178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf17ea-5576-434d-b2b5-26ab9b1eeeea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:14.789747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-afaf17ea-5576-434d-b2b5-26ab9b1eeeea None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:14.790982 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-afaf17ea-5576-434d-b2b5-26ab9b1eeeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:14.839442 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918a2f1-d8e5-4cc5-9fc1-68339297695b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:14.862845 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1101/2208] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:14.866638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:14.877408 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf17ea-5576-434d-b2b5-26ab9b1eeeea None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:14.879519 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-afaf17ea-5576-434d-b2b5-26ab9b1eeeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:14.932640 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef8c6a-5928-4232-a54c-3c3266eb8bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:14.933363 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef8c6a-5928-4232-a54c-3c3266eb8bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:14.933663 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef8c6a-5928-4232-a54c-3c3266eb8bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:14.933937 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef8c6a-5928-4232-a54c-3c3266eb8bd0 admin admin] PATH_INFO: `/v3/users/e693eb8b5dd84f5ab02cf3956ca2ff76` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:14.935223 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef8c6a-5928-4232-a54c-3c3266eb8bd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e693eb8b5dd84f5ab02cf3956ca2ff76)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:14.937660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef8c6a-5928-4232-a54c-3c3266eb8bd0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:14.958473 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf17ea-5576-434d-b2b5-26ab9b1eeeea None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:14.963358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271dc089-baf6-47a3-a3be-af5dededd7d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:14.964318 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271dc089-baf6-47a3-a3be-af5dededd7d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:14.964541 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271dc089-baf6-47a3-a3be-af5dededd7d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:14.964883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271dc089-baf6-47a3-a3be-af5dededd7d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:14.976074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271dc089-baf6-47a3-a3be-af5dededd7d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:14.977899 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271dc089-baf6-47a3-a3be-af5dededd7d6 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:14.988581 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1102/2209] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:02:15.000271 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-05ef8c6a-5928-4232-a54c-3c3266eb8bd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e693eb8b5dd84f5ab02cf3956ca2ff76'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:15.008821 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-05ef8c6a-5928-4232-a54c-3c3266eb8bd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e693eb8b5dd84f5ab02cf3956ca2ff76'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:15.009645 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-05ef8c6a-5928-4232-a54c-3c3266eb8bd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e693eb8b5dd84f5ab02cf3956ca2ff76'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:15.018380 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-05ef8c6a-5928-4232-a54c-3c3266eb8bd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e693eb8b5dd84f5ab02cf3956ca2ff76'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:15.019189 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-05ef8c6a-5928-4232-a54c-3c3266eb8bd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e693eb8b5dd84f5ab02cf3956ca2ff76'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:15.036921 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1108/2210] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:02:14 2022] DELETE /identity/v3/users/e693eb8b5dd84f5ab02cf3956ca2ff76 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:15.132956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271dc089-baf6-47a3-a3be-af5dededd7d6 service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:15.134432 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-271dc089-baf6-47a3-a3be-af5dededd7d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:15.154811 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271dc089-baf6-47a3-a3be-af5dededd7d6 service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:15.271351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e38630-6999-47ae-bb69-988beba7dbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:15.272146 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e38630-6999-47ae-bb69-988beba7dbf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:15.272561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e38630-6999-47ae-bb69-988beba7dbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:15.272976 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e38630-6999-47ae-bb69-988beba7dbf5 admin admin] PATH_INFO: `/v3/projects/f025fef40678400ca3b025db20fe3f30` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:15.274270 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e38630-6999-47ae-bb69-988beba7dbf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f025fef40678400ca3b025db20fe3f30)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:15.287979 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e38630-6999-47ae-bb69-988beba7dbf5 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:15.302443 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:15.389806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f0e38630-6999-47ae-bb69-988beba7dbf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f025fef40678400ca3b025db20fe3f30', 'request_id': 'req-f0e38630-6999-47ae-bb69-988beba7dbf5'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:15.401674 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f0e38630-6999-47ae-bb69-988beba7dbf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f025fef40678400ca3b025db20fe3f30', 'request_id': 'req-f0e38630-6999-47ae-bb69-988beba7dbf5'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:15.411740 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f0e38630-6999-47ae-bb69-988beba7dbf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f025fef40678400ca3b025db20fe3f30', 'request_id': 'req-f0e38630-6999-47ae-bb69-988beba7dbf5'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:15.425448 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f0e38630-6999-47ae-bb69-988beba7dbf5 admin admin] The token cache is being invalidate because project f025fef40678400ca3b025db20fe3f30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:15.425808 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f0e38630-6999-47ae-bb69-988beba7dbf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:15.426603 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f0e38630-6999-47ae-bb69-988beba7dbf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:15.428373 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef8c6a-5928-4232-a54c-3c3266eb8bd0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:15.430314 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-05ef8c6a-5928-4232-a54c-3c3266eb8bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:15.430849 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1103/2211] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:15 2022] DELETE /identity/v3/projects/f025fef40678400ca3b025db20fe3f30 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:15.437129 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e748e84-50a6-4e72-8ef3-40bfe9e71412 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:15.437796 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e748e84-50a6-4e72-8ef3-40bfe9e71412 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:15.438418 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e748e84-50a6-4e72-8ef3-40bfe9e71412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:15.452591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef8c6a-5928-4232-a54c-3c3266eb8bd0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:15.487858 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-6e748e84-50a6-4e72-8ef3-40bfe9e71412 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:15.489231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-6e748e84-50a6-4e72-8ef3-40bfe9e71412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:15.536173 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667c51f-5298-497a-90d7-f27695c81152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:15.537048 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667c51f-5298-497a-90d7-f27695c81152 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:15.537429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667c51f-5298-497a-90d7-f27695c81152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:15.537816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667c51f-5298-497a-90d7-f27695c81152 admin admin] PATH_INFO: `/v3/projects/0a369aad0b444cfcbfad28d50efa1a3a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:15.539526 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3667c51f-5298-497a-90d7-f27695c81152 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a369aad0b444cfcbfad28d50efa1a3a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:15.541654 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3667c51f-5298-497a-90d7-f27695c81152 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:15.599145 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1104/2212] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:15.600525 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e748e84-50a6-4e72-8ef3-40bfe9e71412 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:15.607882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-6e748e84-50a6-4e72-8ef3-40bfe9e71412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:15.611324 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:15.616387 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3667c51f-5298-497a-90d7-f27695c81152 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a369aad0b444cfcbfad28d50efa1a3a', 'request_id': 'req-3667c51f-5298-497a-90d7-f27695c81152'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:15.621546 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e748e84-50a6-4e72-8ef3-40bfe9e71412 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:15.627668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3667c51f-5298-497a-90d7-f27695c81152 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a369aad0b444cfcbfad28d50efa1a3a', 'request_id': 'req-3667c51f-5298-497a-90d7-f27695c81152'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:15.640827 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3667c51f-5298-497a-90d7-f27695c81152 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a369aad0b444cfcbfad28d50efa1a3a', 'request_id': 'req-3667c51f-5298-497a-90d7-f27695c81152'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:15.661779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3667c51f-5298-497a-90d7-f27695c81152 admin admin] The token cache is being invalidate because project 0a369aad0b444cfcbfad28d50efa1a3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:15.663878 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3667c51f-5298-497a-90d7-f27695c81152 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:15.665984 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3667c51f-5298-497a-90d7-f27695c81152 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:15.668101 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1109/2213] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:15 2022] DELETE /identity/v3/projects/0a369aad0b444cfcbfad28d50efa1a3a => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:02:15.670864 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a604ba-f2e8-42a4-8e6d-0e79dfe99fa4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:15.671914 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a604ba-f2e8-42a4-8e6d-0e79dfe99fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:15.672546 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a604ba-f2e8-42a4-8e6d-0e79dfe99fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:15.724768 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c0a604ba-f2e8-42a4-8e6d-0e79dfe99fa4 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:15.728467 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c0a604ba-f2e8-42a4-8e6d-0e79dfe99fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:15.734566 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bcedf1-dabd-4c65-a996-27bc52413b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:15.735142 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcedf1-dabd-4c65-a996-27bc52413b9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:15.735482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcedf1-dabd-4c65-a996-27bc52413b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:15.736268 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcedf1-dabd-4c65-a996-27bc52413b9b admin admin] PATH_INFO: `/v3/projects/bd2ad6590d394a99850423f57311fa26` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:15.740101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bcedf1-dabd-4c65-a996-27bc52413b9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd2ad6590d394a99850423f57311fa26)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:15.740801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1110/2214] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:15.742063 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bcedf1-dabd-4c65-a996-27bc52413b9b admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:15.746413 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283d304-e7a8-4a17-8e0f-5dfbcc00583c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:15.746848 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283d304-e7a8-4a17-8e0f-5dfbcc00583c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:15.748207 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283d304-e7a8-4a17-8e0f-5dfbcc00583c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:15.789878 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5283d304-e7a8-4a17-8e0f-5dfbcc00583c None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:15.792037 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5283d304-e7a8-4a17-8e0f-5dfbcc00583c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:15.807636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1111/2215] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:15.809691 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283d304-e7a8-4a17-8e0f-5dfbcc00583c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:15.825782 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283d304-e7a8-4a17-8e0f-5dfbcc00583c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:15.831938 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd77bb3-77c8-432c-a929-124c8d6acc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:15.832771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd77bb3-77c8-432c-a929-124c8d6acc0d admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:15.833160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd77bb3-77c8-432c-a929-124c8d6acc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:15.833535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd77bb3-77c8-432c-a929-124c8d6acc0d admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:15.834564 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd77bb3-77c8-432c-a929-124c8d6acc0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:02:15.834984 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd77bb3-77c8-432c-a929-124c8d6acc0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:15.837468 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd77bb3-77c8-432c-a929-124c8d6acc0d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:15.863049 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1112/2216] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:02:15.867335 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4d3bf-e53c-417b-92c4-8fc4d3722acc None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:15.869370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4d3bf-e53c-417b-92c4-8fc4d3722acc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:15.869741 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4d3bf-e53c-417b-92c4-8fc4d3722acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:15.871037 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:15.894617 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-93bcedf1-dabd-4c65-a996-27bc52413b9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd2ad6590d394a99850423f57311fa26', 'request_id': 'req-93bcedf1-dabd-4c65-a996-27bc52413b9b'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:15.913810 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-93bcedf1-dabd-4c65-a996-27bc52413b9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd2ad6590d394a99850423f57311fa26', 'request_id': 'req-93bcedf1-dabd-4c65-a996-27bc52413b9b'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:15.919090 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b0c4d3bf-e53c-417b-92c4-8fc4d3722acc None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:15.920392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b0c4d3bf-e53c-417b-92c4-8fc4d3722acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:15.921999 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-93bcedf1-dabd-4c65-a996-27bc52413b9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd2ad6590d394a99850423f57311fa26', 'request_id': 'req-93bcedf1-dabd-4c65-a996-27bc52413b9b'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:15.936178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-93bcedf1-dabd-4c65-a996-27bc52413b9b admin admin] The token cache is being invalidate because project bd2ad6590d394a99850423f57311fa26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:15.936423 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-93bcedf1-dabd-4c65-a996-27bc52413b9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:15.937082 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-93bcedf1-dabd-4c65-a996-27bc52413b9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:15.942366 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1105/2217] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:15 2022] DELETE /identity/v3/projects/bd2ad6590d394a99850423f57311fa26 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:15.943347 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bcedf1-dabd-4c65-a996-27bc52413b9b admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:15.945448 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-93bcedf1-dabd-4c65-a996-27bc52413b9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:16.038547 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1113/2218] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:16.039447 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4d3bf-e53c-417b-92c4-8fc4d3722acc None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:16.041051 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b0c4d3bf-e53c-417b-92c4-8fc4d3722acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:16.054857 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4d3bf-e53c-417b-92c4-8fc4d3722acc None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:16.060114 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b8a4c-5ea7-44b0-a71d-400aafc7f4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:16.060903 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b8a4c-5ea7-44b0-a71d-400aafc7f4db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:16.061246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b8a4c-5ea7-44b0-a71d-400aafc7f4db admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:16.061583 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b8a4c-5ea7-44b0-a71d-400aafc7f4db admin admin] PATH_INFO: `/v3/users/baf46b1f78bf46b5ad632b347d4fb8ea` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:16.062836 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b8a4c-5ea7-44b0-a71d-400aafc7f4db admin admin] RBAC: Authorizing `identity:delete_user(user_id=baf46b1f78bf46b5ad632b347d4fb8ea)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:16.065056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b8a4c-5ea7-44b0-a71d-400aafc7f4db admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:16.069627 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bcedf1-dabd-4c65-a996-27bc52413b9b admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:16.079770 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020554b3-6992-4e71-942f-348123410a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:16.080624 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020554b3-6992-4e71-942f-348123410a9c admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:16.080964 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020554b3-6992-4e71-942f-348123410a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:16.081237 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020554b3-6992-4e71-942f-348123410a9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:16.081948 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020554b3-6992-4e71-942f-348123410a9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:16.083980 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020554b3-6992-4e71-942f-348123410a9c admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:16.119294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1106/2219] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:02:15 2022] POST /identity/v3/projects => generated 375 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:16.120397 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020554b3-6992-4e71-942f-348123410a9c admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:16.133212 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0b0b8a4c-5ea7-44b0-a71d-400aafc7f4db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baf46b1f78bf46b5ad632b347d4fb8ea'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:16.148801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0b0b8a4c-5ea7-44b0-a71d-400aafc7f4db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baf46b1f78bf46b5ad632b347d4fb8ea'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:16.149997 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0b0b8a4c-5ea7-44b0-a71d-400aafc7f4db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baf46b1f78bf46b5ad632b347d4fb8ea'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:16.165904 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0b0b8a4c-5ea7-44b0-a71d-400aafc7f4db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baf46b1f78bf46b5ad632b347d4fb8ea'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:16.167060 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0b0b8a4c-5ea7-44b0-a71d-400aafc7f4db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baf46b1f78bf46b5ad632b347d4fb8ea'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:16.196882 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1114/2220] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:02:16 2022] DELETE /identity/v3/users/baf46b1f78bf46b5ad632b347d4fb8ea => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:16.198146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b8a4c-5ea7-44b0-a71d-400aafc7f4db admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:16.199905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0b0b8a4c-5ea7-44b0-a71d-400aafc7f4db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:16.227024 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:16.310899 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b8a4c-5ea7-44b0-a71d-400aafc7f4db admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:16.318469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a2141-5d08-4a0c-8087-ec0973398c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:16.319701 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a2141-5d08-4a0c-8087-ec0973398c38 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:16.320117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a2141-5d08-4a0c-8087-ec0973398c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:16.320504 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a2141-5d08-4a0c-8087-ec0973398c38 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:16.321923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a2141-5d08-4a0c-8087-ec0973398c38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:16.324648 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a2141-5d08-4a0c-8087-ec0973398c38 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:16.335701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020554b3-6992-4e71-942f-348123410a9c admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:16.340076 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef4e0b-1bc6-4536-a223-69770860c82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:16.340912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef4e0b-1bc6-4536-a223-69770860c82e admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:16.341330 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef4e0b-1bc6-4536-a223-69770860c82e admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:16.341745 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef4e0b-1bc6-4536-a223-69770860c82e admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:16.342611 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef4e0b-1bc6-4536-a223-69770860c82e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:02:16.343075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef4e0b-1bc6-4536-a223-69770860c82e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:16.344916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef4e0b-1bc6-4536-a223-69770860c82e admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:16.360704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1115/2221] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:02:16 2022] POST /identity/v3/users => generated 288 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:16.364905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a2141-5d08-4a0c-8087-ec0973398c38 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:16.370256 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1107/2222] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:02:16.371615 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef4e0b-1bc6-4536-a223-69770860c82e admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:16.373789 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-81ef4e0b-1bc6-4536-a223-69770860c82e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:16.379899 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a2141-5d08-4a0c-8087-ec0973398c38 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:16.389065 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9540aed-0958-4444-a696-582e1850b452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:16.389566 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9540aed-0958-4444-a696-582e1850b452 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:16.389932 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9540aed-0958-4444-a696-582e1850b452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:16.390098 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9540aed-0958-4444-a696-582e1850b452 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:16.390806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9540aed-0958-4444-a696-582e1850b452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:16.391170 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef4e0b-1bc6-4536-a223-69770860c82e admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:16.392588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9540aed-0958-4444-a696-582e1850b452 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:16.400828 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d4e048-d2c6-4c94-9e1f-d56c342fc4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:16.401595 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4e048-d2c6-4c94-9e1f-d56c342fc4cb admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:16.402020 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4e048-d2c6-4c94-9e1f-d56c342fc4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:16.402429 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4e048-d2c6-4c94-9e1f-d56c342fc4cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:16.403312 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d4e048-d2c6-4c94-9e1f-d56c342fc4cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:16.404154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1116/2223] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:02:16.408159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d4e048-d2c6-4c94-9e1f-d56c342fc4cb admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:16.409463 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9540aed-0958-4444-a696-582e1850b452 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:16.433968 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9540aed-0958-4444-a696-582e1850b452 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:16.444505 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1debb9e-04fc-4443-9ece-7c39e5355402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:16.445526 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1debb9e-04fc-4443-9ece-7c39e5355402 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:16.446170 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1debb9e-04fc-4443-9ece-7c39e5355402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:16.446865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1debb9e-04fc-4443-9ece-7c39e5355402 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:16.448632 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1debb9e-04fc-4443-9ece-7c39e5355402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:16.451492 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1debb9e-04fc-4443-9ece-7c39e5355402 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:16.470982 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1117/2224] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:02:16.479344 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1108/2225] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:02:16 2022] POST /identity/v3/projects => generated 389 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:16.479529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1debb9e-04fc-4443-9ece-7c39e5355402 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:16.480668 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d4e048-d2c6-4c94-9e1f-d56c342fc4cb admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:16.487957 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1debb9e-04fc-4443-9ece-7c39e5355402 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:16.516838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:16.566018 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cca87e-7b43-455a-9cd8-1c7ff1445d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:16.567233 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cca87e-7b43-455a-9cd8-1c7ff1445d4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:16.568163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cca87e-7b43-455a-9cd8-1c7ff1445d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:16.568163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cca87e-7b43-455a-9cd8-1c7ff1445d4d admin admin] PATH_INFO: `/v3/projects/f240e3cde14144aeb8e8582c66ed8d8c/users/86cba820b8c148848e165bedcafb577a/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:16.573942 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d4e048-d2c6-4c94-9e1f-d56c342fc4cb admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:16.582581 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5481759-d5d9-4342-9198-30f08d6d4de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:16.583226 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5481759-d5d9-4342-9198-30f08d6d4de8 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:16.583505 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5481759-d5d9-4342-9198-30f08d6d4de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:16.584446 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5481759-d5d9-4342-9198-30f08d6d4de8 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:16.585164 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5481759-d5d9-4342-9198-30f08d6d4de8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:16.587634 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5481759-d5d9-4342-9198-30f08d6d4de8 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:16.597053 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cca87e-7b43-455a-9cd8-1c7ff1445d4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f240e3cde14144aeb8e8582c66ed8d8c, user_id=86cba820b8c148848e165bedcafb577a, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:16.602361 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cca87e-7b43-455a-9cd8-1c7ff1445d4d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:16.612639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1109/2226] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:02:16 2022] POST /identity/v3/users => generated 295 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:16.618657 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5481759-d5d9-4342-9198-30f08d6d4de8 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:16.622101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a5481759-d5d9-4342-9198-30f08d6d4de8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:16.641622 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5481759-d5d9-4342-9198-30f08d6d4de8 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:16.643694 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1118/2227] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:16 2022] PUT /identity/v3/projects/f240e3cde14144aeb8e8582c66ed8d8c/users/86cba820b8c148848e165bedcafb577a/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:16.646627 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cca87e-7b43-455a-9cd8-1c7ff1445d4d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:16.754584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cca87e-7b43-455a-9cd8-1c7ff1445d4d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:16.758122 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ba2f3-ad3a-4c63-80d6-456c7ef6eb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:16.759199 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ba2f3-ad3a-4c63-80d6-456c7ef6eb2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:16.759698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ba2f3-ad3a-4c63-80d6-456c7ef6eb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:16.760072 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ba2f3-ad3a-4c63-80d6-456c7ef6eb2b admin admin] PATH_INFO: `/v3/projects/1d81a060457a4778806d9a4729384bf1` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:16.761493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae90edd-4daa-4a45-a8f2-1fc6a3279d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:16.762258 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae90edd-4daa-4a45-a8f2-1fc6a3279d0f admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:16.762506 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ba2f3-ad3a-4c63-80d6-456c7ef6eb2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d81a060457a4778806d9a4729384bf1)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:16.762739 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae90edd-4daa-4a45-a8f2-1fc6a3279d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:16.763097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae90edd-4daa-4a45-a8f2-1fc6a3279d0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:16.763965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae90edd-4daa-4a45-a8f2-1fc6a3279d0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:16.765687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae90edd-4daa-4a45-a8f2-1fc6a3279d0f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:16.766382 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ba2f3-ad3a-4c63-80d6-456c7ef6eb2b admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:16.777632 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1119/2228] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:16.779477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835ca2f-cc84-40d8-8cdd-fa397a17b877 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:16.779885 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835ca2f-cc84-40d8-8cdd-fa397a17b877 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:16.780256 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835ca2f-cc84-40d8-8cdd-fa397a17b877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:16.789369 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:16.823394 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3835ca2f-cc84-40d8-8cdd-fa397a17b877 None None] MFA Rules not processed for user `86cba820b8c148848e165bedcafb577a`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:16.824962 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3835ca2f-cc84-40d8-8cdd-fa397a17b877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:16.834977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-615ba2f3-ad3a-4c63-80d6-456c7ef6eb2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d81a060457a4778806d9a4729384bf1', 'request_id': 'req-615ba2f3-ad3a-4c63-80d6-456c7ef6eb2b'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:16.842906 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-615ba2f3-ad3a-4c63-80d6-456c7ef6eb2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d81a060457a4778806d9a4729384bf1', 'request_id': 'req-615ba2f3-ad3a-4c63-80d6-456c7ef6eb2b'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:16.849568 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-615ba2f3-ad3a-4c63-80d6-456c7ef6eb2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d81a060457a4778806d9a4729384bf1', 'request_id': 'req-615ba2f3-ad3a-4c63-80d6-456c7ef6eb2b'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:16.860839 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-615ba2f3-ad3a-4c63-80d6-456c7ef6eb2b admin admin] The token cache is being invalidate because project 1d81a060457a4778806d9a4729384bf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:16.861137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-615ba2f3-ad3a-4c63-80d6-456c7ef6eb2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:16.861858 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-615ba2f3-ad3a-4c63-80d6-456c7ef6eb2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:16.863464 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1110/2229] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:16 2022] DELETE /identity/v3/projects/1d81a060457a4778806d9a4729384bf1 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:02:16.864275 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ba2f3-ad3a-4c63-80d6-456c7ef6eb2b admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:16.865582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-615ba2f3-ad3a-4c63-80d6-456c7ef6eb2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:16.895770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1120/2230] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:16.897081 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773d0f6-d6cd-4496-ab3f-f575e984965f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:16.897446 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773d0f6-d6cd-4496-ab3f-f575e984965f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:16.897724 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773d0f6-d6cd-4496-ab3f-f575e984965f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:16.928908 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:16.932786 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8773d0f6-d6cd-4496-ab3f-f575e984965f None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:16.934634 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8773d0f6-d6cd-4496-ab3f-f575e984965f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:16.955679 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ba2f3-ad3a-4c63-80d6-456c7ef6eb2b admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:16.961416 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c9640-b3ea-429a-ba78-c2bba00b5a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:16.962056 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c9640-b3ea-429a-ba78-c2bba00b5a42 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:16.962368 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c9640-b3ea-429a-ba78-c2bba00b5a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:16.962641 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c9640-b3ea-429a-ba78-c2bba00b5a42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:16.963362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c9640-b3ea-429a-ba78-c2bba00b5a42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:16.965029 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c9640-b3ea-429a-ba78-c2bba00b5a42 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:16.981191 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1111/2231] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:16.984733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c9640-b3ea-429a-ba78-c2bba00b5a42 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:16.985826 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-914c9640-b3ea-429a-ba78-c2bba00b5a42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:17.015040 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1121/2232] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:17.016477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773d0f6-d6cd-4496-ab3f-f575e984965f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:17.027985 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773d0f6-d6cd-4496-ab3f-f575e984965f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:17.034935 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fcc154-fd5b-4c1d-b0e4-79ff1804caa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:17.036329 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcc154-fd5b-4c1d-b0e4-79ff1804caa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:17.036706 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcc154-fd5b-4c1d-b0e4-79ff1804caa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:17.037054 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcc154-fd5b-4c1d-b0e4-79ff1804caa6 admin admin] PATH_INFO: `/v3/projects/33739a601f854277ae1931e8c4cd14aa/users/b9a38fcd74ca4a358ec6f580d0351b28/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:17.058156 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fcc154-fd5b-4c1d-b0e4-79ff1804caa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33739a601f854277ae1931e8c4cd14aa, user_id=b9a38fcd74ca4a358ec6f580d0351b28, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:17.061307 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fcc154-fd5b-4c1d-b0e4-79ff1804caa6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:17.080308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c9640-b3ea-429a-ba78-c2bba00b5a42 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:17.091650 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1122/2233] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:17 2022] PUT /identity/v3/projects/33739a601f854277ae1931e8c4cd14aa/users/b9a38fcd74ca4a358ec6f580d0351b28/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:17.094649 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b7bcb-b811-4fd7-963e-51c1d5726dcd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:17.097645 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b7bcb-b811-4fd7-963e-51c1d5726dcd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:17.099048 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b7bcb-b811-4fd7-963e-51c1d5726dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:17.158518 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7c6b7bcb-b811-4fd7-963e-51c1d5726dcd None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:17.160344 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7c6b7bcb-b811-4fd7-963e-51c1d5726dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:17.216698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b057d71-db18-4956-95fd-420012af6594 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6946a788d2ef49ff873ca8e81990527e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:17.217362 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b057d71-db18-4956-95fd-420012af6594 service glance] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:17.217769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b057d71-db18-4956-95fd-420012af6594 service glance] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:17.218057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b057d71-db18-4956-95fd-420012af6594 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:17.274789 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1123/2234] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:17.276058 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13a61e-c430-4f4f-a6e4-caee397de470 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:17.276344 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13a61e-c430-4f4f-a6e4-caee397de470 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:17.276617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13a61e-c430-4f4f-a6e4-caee397de470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:17.316170 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b057d71-db18-4956-95fd-420012af6594 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:17.317947 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b057d71-db18-4956-95fd-420012af6594 service glance] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:17.321174 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:17.341456 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fd13a61e-c430-4f4f-a6e4-caee397de470 None None] MFA Rules not processed for user `b9a38fcd74ca4a358ec6f580d0351b28`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:17.343321 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1112/2235] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:02:17.344944 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fd13a61e-c430-4f4f-a6e4-caee397de470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:17.364381 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b057d71-db18-4956-95fd-420012af6594 service glance] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:17.386206 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b057d71-db18-4956-95fd-420012af6594 service glance] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:17.395357 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a146c2-09aa-41fd-a653-780a23b99466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:17.396959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a146c2-09aa-41fd-a653-780a23b99466 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:17.396959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a146c2-09aa-41fd-a653-780a23b99466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:17.397448 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a146c2-09aa-41fd-a653-780a23b99466 admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:17.399126 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a146c2-09aa-41fd-a653-780a23b99466 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:02:17.399886 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a146c2-09aa-41fd-a653-780a23b99466 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:17.403461 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a146c2-09aa-41fd-a653-780a23b99466 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:17.419697 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1113/2236] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:02:17.424194 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a146c2-09aa-41fd-a653-780a23b99466 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:17.437074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a146c2-09aa-41fd-a653-780a23b99466 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:17.443887 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029f628-5c20-4981-a5d5-c7ad030c93c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:17.444864 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029f628-5c20-4981-a5d5-c7ad030c93c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:17.445309 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029f628-5c20-4981-a5d5-c7ad030c93c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:17.445856 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029f628-5c20-4981-a5d5-c7ad030c93c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:17.446475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e029f628-5c20-4981-a5d5-c7ad030c93c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:17.449599 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e029f628-5c20-4981-a5d5-c7ad030c93c0 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:17.453911 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1124/2237] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:17.466246 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13a61e-c430-4f4f-a6e4-caee397de470 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:17.467946 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fd13a61e-c430-4f4f-a6e4-caee397de470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:17.484339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1114/2238] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:02:17 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:17.488531 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029f628-5c20-4981-a5d5-c7ad030c93c0 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:17.503078 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:17.550777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13a61e-c430-4f4f-a6e4-caee397de470 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:17.568001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029f628-5c20-4981-a5d5-c7ad030c93c0 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:17.576058 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e1c1de-8168-4ae8-98c5-78529cf41d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:17.577582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1c1de-8168-4ae8-98c5-78529cf41d1a admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:17.578069 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1c1de-8168-4ae8-98c5-78529cf41d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:17.578396 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1c1de-8168-4ae8-98c5-78529cf41d1a admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:17.579203 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e1c1de-8168-4ae8-98c5-78529cf41d1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:17.581353 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e1c1de-8168-4ae8-98c5-78529cf41d1a admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:17.610007 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1115/2239] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:02:17 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:02:17.614248 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e1c1de-8168-4ae8-98c5-78529cf41d1a admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:17.616305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d447f-bafb-4f5e-a657-762c5f72b5c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:17.616997 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d447f-bafb-4f5e-a657-762c5f72b5c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:17.617292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d447f-bafb-4f5e-a657-762c5f72b5c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:17.617564 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d447f-bafb-4f5e-a657-762c5f72b5c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:17.630105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e1c1de-8168-4ae8-98c5-78529cf41d1a admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:17.641447 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a073f48-b2f9-4e46-a398-d97991206780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:17.642267 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a073f48-b2f9-4e46-a398-d97991206780 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:17.642682 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a073f48-b2f9-4e46-a398-d97991206780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:17.643356 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a073f48-b2f9-4e46-a398-d97991206780 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:17.643944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a073f48-b2f9-4e46-a398-d97991206780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:17.646394 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a073f48-b2f9-4e46-a398-d97991206780 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:17.658334 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1116/2240] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:02:17.662775 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a073f48-b2f9-4e46-a398-d97991206780 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:17.675078 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a073f48-b2f9-4e46-a398-d97991206780 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:17.681298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5890e1e-f73a-438e-807f-f948fe53e46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:17.682453 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5890e1e-f73a-438e-807f-f948fe53e46a admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:17.682734 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5890e1e-f73a-438e-807f-f948fe53e46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:17.682830 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d447f-bafb-4f5e-a657-762c5f72b5c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:17.683068 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5890e1e-f73a-438e-807f-f948fe53e46a admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:17.683801 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5890e1e-f73a-438e-807f-f948fe53e46a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:17.684595 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d447f-bafb-4f5e-a657-762c5f72b5c9 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:17.685867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5890e1e-f73a-438e-807f-f948fe53e46a admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:17.698199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1125/2241] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:02:17.702478 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1117/2242] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:17.703959 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d447f-bafb-4f5e-a657-762c5f72b5c9 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:17.716946 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d447f-bafb-4f5e-a657-762c5f72b5c9 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:17.723605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb73112-2bcc-43bf-8034-0a272575a8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:17.724341 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb73112-2bcc-43bf-8034-0a272575a8bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:17.724652 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb73112-2bcc-43bf-8034-0a272575a8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:17.725042 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb73112-2bcc-43bf-8034-0a272575a8bd admin admin] PATH_INFO: `/v3/projects/9229dbc411944459b686adf3717cbfb5/users/c4955a6fef784af0aafd8c9676d42828/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:17.735218 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:17.746506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb73112-2bcc-43bf-8034-0a272575a8bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9229dbc411944459b686adf3717cbfb5, user_id=c4955a6fef784af0aafd8c9676d42828, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:17.749450 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb73112-2bcc-43bf-8034-0a272575a8bd admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:17.774315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1126/2243] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:17 2022] PUT /identity/v3/projects/9229dbc411944459b686adf3717cbfb5/users/c4955a6fef784af0aafd8c9676d42828/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:02:17.776829 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b492e-2c49-4cc7-897d-a0bce953fbd5 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:17.777118 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b492e-2c49-4cc7-897d-a0bce953fbd5 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:17.777504 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b492e-2c49-4cc7-897d-a0bce953fbd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:17.813360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-448b492e-2c49-4cc7-897d-a0bce953fbd5 None None] MFA Rules not processed for user `c4955a6fef784af0aafd8c9676d42828`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:17.815259 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-448b492e-2c49-4cc7-897d-a0bce953fbd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:17.913154 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1118/2244] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:17.918363 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb73112-2bcc-43bf-8034-0a272575a8bd admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:17.992344 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb73112-2bcc-43bf-8034-0a272575a8bd admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:17.998532 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a5d46-464f-4f02-ba6a-47720fe33927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:17.999228 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a5d46-464f-4f02-ba6a-47720fe33927 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:17.999518 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a5d46-464f-4f02-ba6a-47720fe33927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:17.999834 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a5d46-464f-4f02-ba6a-47720fe33927 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:18.000526 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a5d46-464f-4f02-ba6a-47720fe33927 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:18.002543 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a5d46-464f-4f02-ba6a-47720fe33927 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:18.051692 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1127/2245] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:02:17 2022] POST /identity/v3/projects => generated 371 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:02:18.054253 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b492e-2c49-4cc7-897d-a0bce953fbd5 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:18.073698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b492e-2c49-4cc7-897d-a0bce953fbd5 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:18.148321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a5d46-464f-4f02-ba6a-47720fe33927 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:18.150853 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3e8a5d46-464f-4f02-ba6a-47720fe33927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:18.158711 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39d82c-3afd-4ef5-861f-1c5b09940654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:18.159502 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39d82c-3afd-4ef5-861f-1c5b09940654 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:18.159830 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39d82c-3afd-4ef5-861f-1c5b09940654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:18.160105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39d82c-3afd-4ef5-861f-1c5b09940654 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:18.160928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39d82c-3afd-4ef5-861f-1c5b09940654 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:18.163353 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39d82c-3afd-4ef5-861f-1c5b09940654 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:18.179515 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:18.185770 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1119/2246] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:02:18 2022] POST /identity/v3/users => generated 292 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:18.189279 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39d82c-3afd-4ef5-861f-1c5b09940654 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:18.200983 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39d82c-3afd-4ef5-861f-1c5b09940654 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:18.207410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541be49-3816-46c0-8b9e-86a775cc3982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:18.208069 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541be49-3816-46c0-8b9e-86a775cc3982 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:18.208475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541be49-3816-46c0-8b9e-86a775cc3982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:18.208754 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541be49-3816-46c0-8b9e-86a775cc3982 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:18.209471 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0541be49-3816-46c0-8b9e-86a775cc3982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:18.211150 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0541be49-3816-46c0-8b9e-86a775cc3982 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:18.220721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a5d46-464f-4f02-ba6a-47720fe33927 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:18.222120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1120/2247] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:02:18.227967 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d43d66-e2b3-4d7b-9ce7-c5c5fae2b8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:18.228510 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d43d66-e2b3-4d7b-9ce7-c5c5fae2b8d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:18.228705 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541be49-3816-46c0-8b9e-86a775cc3982 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:18.228831 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d43d66-e2b3-4d7b-9ce7-c5c5fae2b8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:18.229060 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d43d66-e2b3-4d7b-9ce7-c5c5fae2b8d0 admin admin] PATH_INFO: `/v3/users/815705d191904e6f977b68273f59b66a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:18.230253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d43d66-e2b3-4d7b-9ce7-c5c5fae2b8d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=815705d191904e6f977b68273f59b66a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:18.232250 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d43d66-e2b3-4d7b-9ce7-c5c5fae2b8d0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:18.245030 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541be49-3816-46c0-8b9e-86a775cc3982 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:18.250755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4df2c2-1bca-4ba7-a436-8351841d6b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:18.251445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4df2c2-1bca-4ba7-a436-8351841d6b03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:18.251755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4df2c2-1bca-4ba7-a436-8351841d6b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:18.252029 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4df2c2-1bca-4ba7-a436-8351841d6b03 admin admin] PATH_INFO: `/v3/projects/34a9729e903943c19e3413a0ffeb7840/users/8ad88ce143334f9ebda4498a9fdcaa0e/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:18.269368 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4df2c2-1bca-4ba7-a436-8351841d6b03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34a9729e903943c19e3413a0ffeb7840, user_id=8ad88ce143334f9ebda4498a9fdcaa0e, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:18.272071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4df2c2-1bca-4ba7-a436-8351841d6b03 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:18.288930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1121/2248] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:18 2022] PUT /identity/v3/projects/34a9729e903943c19e3413a0ffeb7840/users/8ad88ce143334f9ebda4498a9fdcaa0e/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:18.293306 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4df2c2-1bca-4ba7-a436-8351841d6b03 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:18.300652 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e4d43d66-e2b3-4d7b-9ce7-c5c5fae2b8d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '815705d191904e6f977b68273f59b66a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:18.313298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e4d43d66-e2b3-4d7b-9ce7-c5c5fae2b8d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '815705d191904e6f977b68273f59b66a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:18.314655 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e4d43d66-e2b3-4d7b-9ce7-c5c5fae2b8d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '815705d191904e6f977b68273f59b66a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:18.331235 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e4d43d66-e2b3-4d7b-9ce7-c5c5fae2b8d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '815705d191904e6f977b68273f59b66a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:18.332164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e4d43d66-e2b3-4d7b-9ce7-c5c5fae2b8d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '815705d191904e6f977b68273f59b66a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:18.344986 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:18.363652 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1128/2249] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:02:18 2022] DELETE /identity/v3/users/815705d191904e6f977b68273f59b66a => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:18.367399 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7353f0-ac8b-4f44-a305-36c016411276 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:18.367399 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7353f0-ac8b-4f44-a305-36c016411276 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:18.367399 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7353f0-ac8b-4f44-a305-36c016411276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:18.388867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4df2c2-1bca-4ba7-a436-8351841d6b03 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:18.398643 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7b7353f0-ac8b-4f44-a305-36c016411276 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:18.400854 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7b7353f0-ac8b-4f44-a305-36c016411276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:18.475164 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e1c147-51e8-464d-b948-f58d14bfe6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:18.475997 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1c147-51e8-464d-b948-f58d14bfe6bf admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:18.476537 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1c147-51e8-464d-b948-f58d14bfe6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:18.476537 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1c147-51e8-464d-b948-f58d14bfe6bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:18.478200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1129/2250] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:18.488144 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7353f0-ac8b-4f44-a305-36c016411276 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:18.490428 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e1c147-51e8-464d-b948-f58d14bfe6bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:18.490796 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7b7353f0-ac8b-4f44-a305-36c016411276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:18.492615 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e1c147-51e8-464d-b948-f58d14bfe6bf admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:18.517222 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1122/2251] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:18.518457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e1c147-51e8-464d-b948-f58d14bfe6bf admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:18.520873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c9e1c147-51e8-464d-b948-f58d14bfe6bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:18.543119 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e1c147-51e8-464d-b948-f58d14bfe6bf admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:18.553560 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90c945-368f-409d-81b3-5f2e3d9c8b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:18.554033 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90c945-368f-409d-81b3-5f2e3d9c8b78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:18.554346 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90c945-368f-409d-81b3-5f2e3d9c8b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:18.554963 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90c945-368f-409d-81b3-5f2e3d9c8b78 admin admin] PATH_INFO: `/v3/projects/34a9729e903943c19e3413a0ffeb7840/users/8ad88ce143334f9ebda4498a9fdcaa0e/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:18.558189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad90c945-368f-409d-81b3-5f2e3d9c8b78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34a9729e903943c19e3413a0ffeb7840, user_id=8ad88ce143334f9ebda4498a9fdcaa0e, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:18.563211 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad90c945-368f-409d-81b3-5f2e3d9c8b78 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:18.590372 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1123/2252] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:18 2022] PUT /identity/v3/projects/34a9729e903943c19e3413a0ffeb7840/users/8ad88ce143334f9ebda4498a9fdcaa0e/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:18.599792 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7353f0-ac8b-4f44-a305-36c016411276 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:18.603930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30280dd3-7162-49ac-be80-a38bd0a5e099 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:18.604292 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30280dd3-7162-49ac-be80-a38bd0a5e099 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:18.604638 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30280dd3-7162-49ac-be80-a38bd0a5e099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:18.657965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-30280dd3-7162-49ac-be80-a38bd0a5e099 None None] MFA Rules not processed for user `8ad88ce143334f9ebda4498a9fdcaa0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:18.661716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-30280dd3-7162-49ac-be80-a38bd0a5e099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:18.700102 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bfbb9a-6dbc-4342-904d-86072c676962 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:18.700791 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfbb9a-6dbc-4342-904d-86072c676962 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:18.701263 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfbb9a-6dbc-4342-904d-86072c676962 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:18.701545 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfbb9a-6dbc-4342-904d-86072c676962 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:18.730483 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:18.759539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1124/2253] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:18 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:02:18.774594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30280dd3-7162-49ac-be80-a38bd0a5e099 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:18.776050 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-30280dd3-7162-49ac-be80-a38bd0a5e099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:18.800320 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bfbb9a-6dbc-4342-904d-86072c676962 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:18.803307 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bfbb9a-6dbc-4342-904d-86072c676962 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:18.807826 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:18.816276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1130/2254] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:02:18.842838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30280dd3-7162-49ac-be80-a38bd0a5e099 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:18.847770 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4650bc2-35f2-421f-abd7-b8248c4d8e9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:18.848702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4650bc2-35f2-421f-abd7-b8248c4d8e9f service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:18.849165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4650bc2-35f2-421f-abd7-b8248c4d8e9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:18.849441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4650bc2-35f2-421f-abd7-b8248c4d8e9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:18.863813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4650bc2-35f2-421f-abd7-b8248c4d8e9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:18.865453 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4650bc2-35f2-421f-abd7-b8248c4d8e9f service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:18.875368 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1125/2255] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:18 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:02:19.112995 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bfbb9a-6dbc-4342-904d-86072c676962 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:19.116412 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-91bfbb9a-6dbc-4342-904d-86072c676962 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:19.139061 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bfbb9a-6dbc-4342-904d-86072c676962 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:19.145572 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77ccb3-037f-40d0-bcd4-9c3ba4d580ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:19.146243 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77ccb3-037f-40d0-bcd4-9c3ba4d580ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:19.147043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77ccb3-037f-40d0-bcd4-9c3ba4d580ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:19.148148 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77ccb3-037f-40d0-bcd4-9c3ba4d580ce admin admin] PATH_INFO: `/v3/projects/42652e9232b1477c9342c6d618ba1ad7` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:19.149493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d77ccb3-037f-40d0-bcd4-9c3ba4d580ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=42652e9232b1477c9342c6d618ba1ad7)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:19.151591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d77ccb3-037f-40d0-bcd4-9c3ba4d580ce admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:19.215193 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2d77ccb3-037f-40d0-bcd4-9c3ba4d580ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42652e9232b1477c9342c6d618ba1ad7', 'request_id': 'req-2d77ccb3-037f-40d0-bcd4-9c3ba4d580ce'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:19.226365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2d77ccb3-037f-40d0-bcd4-9c3ba4d580ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42652e9232b1477c9342c6d618ba1ad7', 'request_id': 'req-2d77ccb3-037f-40d0-bcd4-9c3ba4d580ce'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:19.239178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2d77ccb3-037f-40d0-bcd4-9c3ba4d580ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42652e9232b1477c9342c6d618ba1ad7', 'request_id': 'req-2d77ccb3-037f-40d0-bcd4-9c3ba4d580ce'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:19.258027 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2d77ccb3-037f-40d0-bcd4-9c3ba4d580ce admin admin] The token cache is being invalidate because project 42652e9232b1477c9342c6d618ba1ad7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:19.258501 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2d77ccb3-037f-40d0-bcd4-9c3ba4d580ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:19.259978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2d77ccb3-037f-40d0-bcd4-9c3ba4d580ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:19.262386 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1131/2256] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:19 2022] DELETE /identity/v3/projects/42652e9232b1477c9342c6d618ba1ad7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:19.958860 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4650bc2-35f2-421f-abd7-b8248c4d8e9f service cinder] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:19.970356 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4650bc2-35f2-421f-abd7-b8248c4d8e9f service cinder] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:20.036233 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982f08b-c36a-4cdd-a94f-5d98db4cc957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:20.036233 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982f08b-c36a-4cdd-a94f-5d98db4cc957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:20.036791 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982f08b-c36a-4cdd-a94f-5d98db4cc957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:20.037230 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982f08b-c36a-4cdd-a94f-5d98db4cc957 admin admin] PATH_INFO: `/v3/users/c4955a6fef784af0aafd8c9676d42828` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:20.038536 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982f08b-c36a-4cdd-a94f-5d98db4cc957 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4955a6fef784af0aafd8c9676d42828)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:20.042603 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982f08b-c36a-4cdd-a94f-5d98db4cc957 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:20.094667 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-4982f08b-c36a-4cdd-a94f-5d98db4cc957 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4955a6fef784af0aafd8c9676d42828'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:20.102432 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-4982f08b-c36a-4cdd-a94f-5d98db4cc957 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4955a6fef784af0aafd8c9676d42828'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:20.111345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-4982f08b-c36a-4cdd-a94f-5d98db4cc957 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4955a6fef784af0aafd8c9676d42828'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:20.112075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-4982f08b-c36a-4cdd-a94f-5d98db4cc957 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4955a6fef784af0aafd8c9676d42828'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:20.112843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-4982f08b-c36a-4cdd-a94f-5d98db4cc957 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4955a6fef784af0aafd8c9676d42828'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:20.132666 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1126/2257] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:02:19 2022] DELETE /identity/v3/users/c4955a6fef784af0aafd8c9676d42828 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:20.137506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422afc1e-e05f-46d9-b483-1fc384b1012f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:20.138050 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422afc1e-e05f-46d9-b483-1fc384b1012f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:20.138050 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422afc1e-e05f-46d9-b483-1fc384b1012f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:20.158580 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-422afc1e-e05f-46d9-b483-1fc384b1012f None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:20.159762 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-422afc1e-e05f-46d9-b483-1fc384b1012f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:20.236010 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1132/2258] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:20.246240 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982f08b-c36a-4cdd-a94f-5d98db4cc957 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:20.247801 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-4982f08b-c36a-4cdd-a94f-5d98db4cc957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:20.253441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:20.321226 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982f08b-c36a-4cdd-a94f-5d98db4cc957 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:20.327128 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c27363-a11c-4046-9d4b-35acc86fa4b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:20.328035 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c27363-a11c-4046-9d4b-35acc86fa4b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:20.328313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c27363-a11c-4046-9d4b-35acc86fa4b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:20.328592 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c27363-a11c-4046-9d4b-35acc86fa4b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:20.340659 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c27363-a11c-4046-9d4b-35acc86fa4b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:20.342132 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c27363-a11c-4046-9d4b-35acc86fa4b5 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:20.353564 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1127/2259] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:02:20.654491 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422afc1e-e05f-46d9-b483-1fc384b1012f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:20.656700 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-422afc1e-e05f-46d9-b483-1fc384b1012f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:20.670548 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422afc1e-e05f-46d9-b483-1fc384b1012f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:20.690274 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd83c9-f7ca-4bb0-a53e-3c825a5f8641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:20.690945 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd83c9-f7ca-4bb0-a53e-3c825a5f8641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:20.691228 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd83c9-f7ca-4bb0-a53e-3c825a5f8641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:20.691501 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd83c9-f7ca-4bb0-a53e-3c825a5f8641 admin admin] PATH_INFO: `/v3/projects/9229dbc411944459b686adf3717cbfb5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:20.700138 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd83c9-f7ca-4bb0-a53e-3c825a5f8641 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9229dbc411944459b686adf3717cbfb5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:20.706105 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd83c9-f7ca-4bb0-a53e-3c825a5f8641 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:20.786242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-81bd83c9-f7ca-4bb0-a53e-3c825a5f8641 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9229dbc411944459b686adf3717cbfb5', 'request_id': 'req-81bd83c9-f7ca-4bb0-a53e-3c825a5f8641'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:20.793531 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-81bd83c9-f7ca-4bb0-a53e-3c825a5f8641 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9229dbc411944459b686adf3717cbfb5', 'request_id': 'req-81bd83c9-f7ca-4bb0-a53e-3c825a5f8641'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:20.802677 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-81bd83c9-f7ca-4bb0-a53e-3c825a5f8641 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9229dbc411944459b686adf3717cbfb5', 'request_id': 'req-81bd83c9-f7ca-4bb0-a53e-3c825a5f8641'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:20.815118 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-81bd83c9-f7ca-4bb0-a53e-3c825a5f8641 admin admin] The token cache is being invalidate because project 9229dbc411944459b686adf3717cbfb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:20.815284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-81bd83c9-f7ca-4bb0-a53e-3c825a5f8641 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:20.815991 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-81bd83c9-f7ca-4bb0-a53e-3c825a5f8641 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:20.817765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1133/2260] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:20 2022] DELETE /identity/v3/projects/9229dbc411944459b686adf3717cbfb5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:20.821731 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c27363-a11c-4046-9d4b-35acc86fa4b5 service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:20.832942 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c27363-a11c-4046-9d4b-35acc86fa4b5 service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:20.902273 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f03875-3ef1-4937-bced-1890329ff93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:20.903032 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f03875-3ef1-4937-bced-1890329ff93c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:20.903317 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f03875-3ef1-4937-bced-1890329ff93c admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:20.903729 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f03875-3ef1-4937-bced-1890329ff93c admin admin] PATH_INFO: `/v3/users/8ad88ce143334f9ebda4498a9fdcaa0e` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:20.904804 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f03875-3ef1-4937-bced-1890329ff93c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ad88ce143334f9ebda4498a9fdcaa0e)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:20.906801 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f03875-3ef1-4937-bced-1890329ff93c admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:20.958326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-74f03875-3ef1-4937-bced-1890329ff93c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ad88ce143334f9ebda4498a9fdcaa0e'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:20.965958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-74f03875-3ef1-4937-bced-1890329ff93c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ad88ce143334f9ebda4498a9fdcaa0e'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:20.973923 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-74f03875-3ef1-4937-bced-1890329ff93c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ad88ce143334f9ebda4498a9fdcaa0e'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:20.974634 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-74f03875-3ef1-4937-bced-1890329ff93c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ad88ce143334f9ebda4498a9fdcaa0e'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:20.975365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-74f03875-3ef1-4937-bced-1890329ff93c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ad88ce143334f9ebda4498a9fdcaa0e'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:20.992895 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1128/2261] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:02:20 2022] DELETE /identity/v3/users/8ad88ce143334f9ebda4498a9fdcaa0e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:02:21.301375 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd83c9-f7ca-4bb0-a53e-3c825a5f8641 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:21.303311 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-81bd83c9-f7ca-4bb0-a53e-3c825a5f8641 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:21.314729 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:21.319976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd83c9-f7ca-4bb0-a53e-3c825a5f8641 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:21.404245 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b825648-7ad8-4a11-afe1-be9c6f16375d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:21.405022 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b825648-7ad8-4a11-afe1-be9c6f16375d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:21.405537 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b825648-7ad8-4a11-afe1-be9c6f16375d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:21.405911 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b825648-7ad8-4a11-afe1-be9c6f16375d admin admin] PATH_INFO: `/v3/projects/34a9729e903943c19e3413a0ffeb7840` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:21.407329 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b825648-7ad8-4a11-afe1-be9c6f16375d admin admin] RBAC: Authorizing `identity:delete_project(project_id=34a9729e903943c19e3413a0ffeb7840)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:21.410718 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b825648-7ad8-4a11-afe1-be9c6f16375d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:21.495157 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8b825648-7ad8-4a11-afe1-be9c6f16375d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34a9729e903943c19e3413a0ffeb7840', 'request_id': 'req-8b825648-7ad8-4a11-afe1-be9c6f16375d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:21.502673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8b825648-7ad8-4a11-afe1-be9c6f16375d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34a9729e903943c19e3413a0ffeb7840', 'request_id': 'req-8b825648-7ad8-4a11-afe1-be9c6f16375d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:21.513225 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8b825648-7ad8-4a11-afe1-be9c6f16375d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34a9729e903943c19e3413a0ffeb7840', 'request_id': 'req-8b825648-7ad8-4a11-afe1-be9c6f16375d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:21.527972 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8b825648-7ad8-4a11-afe1-be9c6f16375d admin admin] The token cache is being invalidate because project 34a9729e903943c19e3413a0ffeb7840 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:21.528267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8b825648-7ad8-4a11-afe1-be9c6f16375d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:21.529247 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8b825648-7ad8-4a11-afe1-be9c6f16375d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:21.530923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1134/2262] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:21 2022] DELETE /identity/v3/projects/34a9729e903943c19e3413a0ffeb7840 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:21.536991 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0a04f-4c98-4c4c-904a-64e842aa502b None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:21.537761 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0a04f-4c98-4c4c-904a-64e842aa502b None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:21.538367 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0a04f-4c98-4c4c-904a-64e842aa502b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:21.565698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-e7f0a04f-4c98-4c4c-904a-64e842aa502b None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:21.566929 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-e7f0a04f-4c98-4c4c-904a-64e842aa502b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:21.590382 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:21.633601 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1129/2263] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:21.641692 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2419845-3e23-4421-bf10-9ef5530349e9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:21.641994 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2419845-3e23-4421-bf10-9ef5530349e9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:21.642273 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2419845-3e23-4421-bf10-9ef5530349e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:21.663818 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c2419845-3e23-4421-bf10-9ef5530349e9 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:21.664971 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c2419845-3e23-4421-bf10-9ef5530349e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:21.676850 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1135/2264] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:21.681413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0a04f-4c98-4c4c-904a-64e842aa502b None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:21.691590 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0a04f-4c98-4c4c-904a-64e842aa502b None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:21.696784 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8b806-b4df-44a1-b680-ef12917da1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:21.697432 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8b806-b4df-44a1-b680-ef12917da1b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:21.698072 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8b806-b4df-44a1-b680-ef12917da1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:21.698338 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8b806-b4df-44a1-b680-ef12917da1b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:21.699054 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d8b806-b4df-44a1-b680-ef12917da1b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:02:21.699361 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d8b806-b4df-44a1-b680-ef12917da1b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:21.701049 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d8b806-b4df-44a1-b680-ef12917da1b8 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:21.714518 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1130/2265] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:02:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:02:21.719694 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2419845-3e23-4421-bf10-9ef5530349e9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:21.729949 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2419845-3e23-4421-bf10-9ef5530349e9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:21.734898 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc62952-4073-48c9-babd-5f28fb313b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:21.735612 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc62952-4073-48c9-babd-5f28fb313b7b admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:21.735855 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc62952-4073-48c9-babd-5f28fb313b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:21.736136 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc62952-4073-48c9-babd-5f28fb313b7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:21.736978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc62952-4073-48c9-babd-5f28fb313b7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:21.739063 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc62952-4073-48c9-babd-5f28fb313b7b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:21.790351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1136/2266] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:02:21 2022] POST /identity/v3/projects => generated 381 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:02:21.794487 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8b806-b4df-44a1-b680-ef12917da1b8 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:21.811690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8b806-b4df-44a1-b680-ef12917da1b8 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:21.881413 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb4be9-7abd-42bb-9dbd-2f1d2948b5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:21.882083 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb4be9-7abd-42bb-9dbd-2f1d2948b5e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:21.882372 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb4be9-7abd-42bb-9dbd-2f1d2948b5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:21.882649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb4be9-7abd-42bb-9dbd-2f1d2948b5e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:21.883390 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb4be9-7abd-42bb-9dbd-2f1d2948b5e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:21.885433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb4be9-7abd-42bb-9dbd-2f1d2948b5e2 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:21.909637 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1131/2267] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:02:21 2022] POST /identity/v3/users => generated 291 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:21.915527 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc62952-4073-48c9-babd-5f28fb313b7b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:21.927191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc62952-4073-48c9-babd-5f28fb313b7b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:21.932382 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8abe90d-f7cb-4323-a18a-640ddc9f0518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:21.933110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abe90d-f7cb-4323-a18a-640ddc9f0518 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:21.933456 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abe90d-f7cb-4323-a18a-640ddc9f0518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:21.933791 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abe90d-f7cb-4323-a18a-640ddc9f0518 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:21.934588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8abe90d-f7cb-4323-a18a-640ddc9f0518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:21.936313 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8abe90d-f7cb-4323-a18a-640ddc9f0518 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:21.951417 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1137/2268] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:02:21.953181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb4be9-7abd-42bb-9dbd-2f1d2948b5e2 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:21.965901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb4be9-7abd-42bb-9dbd-2f1d2948b5e2 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:21.975889 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fae3e4-fe20-478c-9300-6e4d260491f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:21.977305 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fae3e4-fe20-478c-9300-6e4d260491f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:21.977845 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fae3e4-fe20-478c-9300-6e4d260491f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:21.978335 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fae3e4-fe20-478c-9300-6e4d260491f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:21.979226 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fae3e4-fe20-478c-9300-6e4d260491f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:21.981134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fae3e4-fe20-478c-9300-6e4d260491f7 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:21.993280 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1132/2269] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:21.997395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8abe90d-f7cb-4323-a18a-640ddc9f0518 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:22.008000 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8abe90d-f7cb-4323-a18a-640ddc9f0518 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:22.012905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48a24c-68a9-44bc-8e05-6df42f7b8d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:22.013931 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48a24c-68a9-44bc-8e05-6df42f7b8d1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:22.014210 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48a24c-68a9-44bc-8e05-6df42f7b8d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:22.014501 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48a24c-68a9-44bc-8e05-6df42f7b8d1b admin admin] PATH_INFO: `/v3/projects/57859da276c34a0bb939ceef00f1245c/users/0881ebb564a540a1b6bf9f61fd70ea22/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:22.030106 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed48a24c-68a9-44bc-8e05-6df42f7b8d1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=57859da276c34a0bb939ceef00f1245c, user_id=0881ebb564a540a1b6bf9f61fd70ea22, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:22.032853 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed48a24c-68a9-44bc-8e05-6df42f7b8d1b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:22.048306 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1138/2270] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:21 2022] PUT /identity/v3/projects/57859da276c34a0bb939ceef00f1245c/users/0881ebb564a540a1b6bf9f61fd70ea22/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:22.060102 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579203d-43cc-404a-b49b-00e0ec1cd248 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:22.060369 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579203d-43cc-404a-b49b-00e0ec1cd248 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:22.060671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579203d-43cc-404a-b49b-00e0ec1cd248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:22.092161 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-7579203d-43cc-404a-b49b-00e0ec1cd248 None None] MFA Rules not processed for user `0881ebb564a540a1b6bf9f61fd70ea22`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:22.093491 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-7579203d-43cc-404a-b49b-00e0ec1cd248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:22.174367 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1133/2271] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:02:22.180377 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48a24c-68a9-44bc-8e05-6df42f7b8d1b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:22.219690 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:22.258817 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48a24c-68a9-44bc-8e05-6df42f7b8d1b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:22.263969 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb79ce9-6926-4d1d-9a98-893c830352f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:22.264610 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb79ce9-6926-4d1d-9a98-893c830352f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:22.264899 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb79ce9-6926-4d1d-9a98-893c830352f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:22.265182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb79ce9-6926-4d1d-9a98-893c830352f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:22.265857 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb79ce9-6926-4d1d-9a98-893c830352f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:22.267827 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb79ce9-6926-4d1d-9a98-893c830352f9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:22.299179 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1139/2272] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:02:22 2022] POST /identity/v3/projects => generated 381 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:22.305608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579203d-43cc-404a-b49b-00e0ec1cd248 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:22.317773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579203d-43cc-404a-b49b-00e0ec1cd248 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:22.344823 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:22.401349 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233c08e-100e-44b1-8dc6-37324cc72ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:22.401987 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233c08e-100e-44b1-8dc6-37324cc72ecc admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:22.402261 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233c08e-100e-44b1-8dc6-37324cc72ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:22.402534 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233c08e-100e-44b1-8dc6-37324cc72ecc admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:22.403223 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233c08e-100e-44b1-8dc6-37324cc72ecc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:22.405205 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233c08e-100e-44b1-8dc6-37324cc72ecc admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:22.428399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1134/2273] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:02:22 2022] POST /identity/v3/users => generated 297 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:22.432745 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb79ce9-6926-4d1d-9a98-893c830352f9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:22.446371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb79ce9-6926-4d1d-9a98-893c830352f9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:22.454248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d061a65-9e70-4bbb-a8ed-03aac246dddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:22.455167 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d061a65-9e70-4bbb-a8ed-03aac246dddc admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:22.455704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d061a65-9e70-4bbb-a8ed-03aac246dddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:22.456194 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d061a65-9e70-4bbb-a8ed-03aac246dddc admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:22.457444 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d061a65-9e70-4bbb-a8ed-03aac246dddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:22.461295 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d061a65-9e70-4bbb-a8ed-03aac246dddc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:22.477010 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1140/2274] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:02:22.482013 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233c08e-100e-44b1-8dc6-37324cc72ecc admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:22.493691 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233c08e-100e-44b1-8dc6-37324cc72ecc admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:22.498946 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f49242-f55f-4121-ab64-4cb30d91597a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:22.499697 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f49242-f55f-4121-ab64-4cb30d91597a admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:22.499978 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f49242-f55f-4121-ab64-4cb30d91597a admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:22.500535 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f49242-f55f-4121-ab64-4cb30d91597a admin admin] PATH_INFO: `/v3/projects/c9171fb681ec49dcbb86ed9aca419650/users/7038898ec63245a8a5f9ae9e0cedf8c7/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:22.519125 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f49242-f55f-4121-ab64-4cb30d91597a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9171fb681ec49dcbb86ed9aca419650, user_id=7038898ec63245a8a5f9ae9e0cedf8c7, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:22.521890 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f49242-f55f-4121-ab64-4cb30d91597a admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:22.538145 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1135/2275] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:22 2022] PUT /identity/v3/projects/c9171fb681ec49dcbb86ed9aca419650/users/7038898ec63245a8a5f9ae9e0cedf8c7/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:22.544971 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d061a65-9e70-4bbb-a8ed-03aac246dddc admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:22.561044 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d061a65-9e70-4bbb-a8ed-03aac246dddc admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:22.625823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0e0ab-3ad8-455a-bf4e-070106839b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:22.626495 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0e0ab-3ad8-455a-bf4e-070106839b3f admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:22.626779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0e0ab-3ad8-455a-bf4e-070106839b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:22.627061 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0e0ab-3ad8-455a-bf4e-070106839b3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:22.627844 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0e0ab-3ad8-455a-bf4e-070106839b3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:22.629475 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0e0ab-3ad8-455a-bf4e-070106839b3f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:22.640269 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1141/2276] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:22.645234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f49242-f55f-4121-ab64-4cb30d91597a admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:22.657260 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f49242-f55f-4121-ab64-4cb30d91597a admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:22.662387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11207e8e-8f12-4323-985a-c1c872b59093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:22.663057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11207e8e-8f12-4323-985a-c1c872b59093 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:22.663364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11207e8e-8f12-4323-985a-c1c872b59093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:22.663681 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11207e8e-8f12-4323-985a-c1c872b59093 admin admin] PATH_INFO: `/v3/projects/c9171fb681ec49dcbb86ed9aca419650/users/7038898ec63245a8a5f9ae9e0cedf8c7/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:22.665732 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11207e8e-8f12-4323-985a-c1c872b59093 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9171fb681ec49dcbb86ed9aca419650, user_id=7038898ec63245a8a5f9ae9e0cedf8c7, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:22.668420 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11207e8e-8f12-4323-985a-c1c872b59093 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:22.680366 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1136/2277] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:22 2022] PUT /identity/v3/projects/c9171fb681ec49dcbb86ed9aca419650/users/7038898ec63245a8a5f9ae9e0cedf8c7/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:22.691756 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376aef4-49f2-48f0-a440-e401ad533cd9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:22.692061 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376aef4-49f2-48f0-a440-e401ad533cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:22.692524 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376aef4-49f2-48f0-a440-e401ad533cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:22.726061 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5376aef4-49f2-48f0-a440-e401ad533cd9 None None] MFA Rules not processed for user `7038898ec63245a8a5f9ae9e0cedf8c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:22.727327 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5376aef4-49f2-48f0-a440-e401ad533cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:22.801378 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1142/2278] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:22 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:22.814120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11207e8e-8f12-4323-985a-c1c872b59093 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:22.816319 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-11207e8e-8f12-4323-985a-c1c872b59093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:22.892780 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11207e8e-8f12-4323-985a-c1c872b59093 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:22.896574 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54321897-c4cb-4078-b552-a450dfb42d7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:22.897274 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54321897-c4cb-4078-b552-a450dfb42d7c service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:22.897555 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54321897-c4cb-4078-b552-a450dfb42d7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:22.897823 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54321897-c4cb-4078-b552-a450dfb42d7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:22.908117 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54321897-c4cb-4078-b552-a450dfb42d7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:22.909571 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54321897-c4cb-4078-b552-a450dfb42d7c service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:22.920170 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1137/2279] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:22 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:02:23.332346 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376aef4-49f2-48f0-a440-e401ad533cd9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:23.343073 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376aef4-49f2-48f0-a440-e401ad533cd9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:23.348455 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4346bc9-6bb1-451f-9ca7-2aa0e23e4b14 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7038898ec63245a8a5f9ae9e0cedf8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9171fb681ec49dcbb86ed9aca419650', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:23.349109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4346bc9-6bb1-451f-9ca7-2aa0e23e4b14 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:23.349402 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4346bc9-6bb1-451f-9ca7-2aa0e23e4b14 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:23.349685 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4346bc9-6bb1-451f-9ca7-2aa0e23e4b14 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:23.350625 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4346bc9-6bb1-451f-9ca7-2aa0e23e4b14 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:02:23.350967 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4346bc9-6bb1-451f-9ca7-2aa0e23e4b14 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:23.352761 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4346bc9-6bb1-451f-9ca7-2aa0e23e4b14 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:23.368856 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1143/2280] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:02:23.373382 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54321897-c4cb-4078-b552-a450dfb42d7c service cinder] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:23.383343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54321897-c4cb-4078-b552-a450dfb42d7c service cinder] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:23.392397 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b913b6f-46a1-49d3-a9dd-3c563fbd3213 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7038898ec63245a8a5f9ae9e0cedf8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9171fb681ec49dcbb86ed9aca419650', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:23.393306 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b913b6f-46a1-49d3-a9dd-3c563fbd3213 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:23.393621 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b913b6f-46a1-49d3-a9dd-3c563fbd3213 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:23.393885 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b913b6f-46a1-49d3-a9dd-3c563fbd3213 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:23.394603 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b913b6f-46a1-49d3-a9dd-3c563fbd3213 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:23.396661 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b913b6f-46a1-49d3-a9dd-3c563fbd3213 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:23.425116 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1138/2281] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:02:23 2022] POST /identity/v3/projects => generated 354 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:02:23.429520 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4346bc9-6bb1-451f-9ca7-2aa0e23e4b14 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:23.440445 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4346bc9-6bb1-451f-9ca7-2aa0e23e4b14 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:23.450316 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:23.503267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d3fde-e59c-4b4b-915b-42ff037d9253 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7038898ec63245a8a5f9ae9e0cedf8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9171fb681ec49dcbb86ed9aca419650', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:23.503847 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d3fde-e59c-4b4b-915b-42ff037d9253 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:23.504131 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d3fde-e59c-4b4b-915b-42ff037d9253 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:23.504433 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d3fde-e59c-4b4b-915b-42ff037d9253 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:23.505300 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d3fde-e59c-4b4b-915b-42ff037d9253 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:02:23.505603 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d3fde-e59c-4b4b-915b-42ff037d9253 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:23.507293 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d3fde-e59c-4b4b-915b-42ff037d9253 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:23.527516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1144/2282] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 05:02:23.570331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b913b6f-46a1-49d3-a9dd-3c563fbd3213 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:23.581618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b913b6f-46a1-49d3-a9dd-3c563fbd3213 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:23.586892 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af4e78-d023-4606-bb8e-69630b41da37 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7038898ec63245a8a5f9ae9e0cedf8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9171fb681ec49dcbb86ed9aca419650', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:23.587565 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af4e78-d023-4606-bb8e-69630b41da37 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:23.587883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af4e78-d023-4606-bb8e-69630b41da37 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:23.588159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af4e78-d023-4606-bb8e-69630b41da37 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] PATH_INFO: `/v3/projects/16c9bcc85a8447a5bae1b7ef7b784cea` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:23.589366 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3af4e78-d023-4606-bb8e-69630b41da37 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] RBAC: Authorizing `identity:delete_project(project_id=16c9bcc85a8447a5bae1b7ef7b784cea)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:23.591305 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3af4e78-d023-4606-bb8e-69630b41da37 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:23.641593 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=90,keystone:INSERT=6,keystone:DELETE=4 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:23.642925 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c3af4e78-d023-4606-bb8e-69630b41da37 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16c9bcc85a8447a5bae1b7ef7b784cea', 'request_id': 'req-c3af4e78-d023-4606-bb8e-69630b41da37'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:23.651832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c3af4e78-d023-4606-bb8e-69630b41da37 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16c9bcc85a8447a5bae1b7ef7b784cea', 'request_id': 'req-c3af4e78-d023-4606-bb8e-69630b41da37'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:23.659493 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c3af4e78-d023-4606-bb8e-69630b41da37 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16c9bcc85a8447a5bae1b7ef7b784cea', 'request_id': 'req-c3af4e78-d023-4606-bb8e-69630b41da37'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:23.671463 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c3af4e78-d023-4606-bb8e-69630b41da37 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] The token cache is being invalidate because project 16c9bcc85a8447a5bae1b7ef7b784cea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:23.671795 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c3af4e78-d023-4606-bb8e-69630b41da37 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:23.672543 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c3af4e78-d023-4606-bb8e-69630b41da37 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:23.674121 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1139/2283] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:23 2022] DELETE /identity/v3/projects/16c9bcc85a8447a5bae1b7ef7b784cea => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:23.876450 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d3fde-e59c-4b4b-915b-42ff037d9253 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:23.878488 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9e9d3fde-e59c-4b4b-915b-42ff037d9253 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:23.966647 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d3fde-e59c-4b4b-915b-42ff037d9253 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:24.027156 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e43789-97c0-4057-8400-4e16f43850d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:24.027862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e43789-97c0-4057-8400-4e16f43850d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:24.028211 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e43789-97c0-4057-8400-4e16f43850d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:24.028552 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e43789-97c0-4057-8400-4e16f43850d0 admin admin] PATH_INFO: `/v3/users/0881ebb564a540a1b6bf9f61fd70ea22` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:24.029819 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e43789-97c0-4057-8400-4e16f43850d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0881ebb564a540a1b6bf9f61fd70ea22)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:24.031847 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e43789-97c0-4057-8400-4e16f43850d0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:24.092743 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af4e78-d023-4606-bb8e-69630b41da37 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:24.102175 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c3af4e78-d023-4606-bb8e-69630b41da37 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:24.123126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-46e43789-97c0-4057-8400-4e16f43850d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0881ebb564a540a1b6bf9f61fd70ea22'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:24.140762 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-46e43789-97c0-4057-8400-4e16f43850d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0881ebb564a540a1b6bf9f61fd70ea22'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:24.141241 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af4e78-d023-4606-bb8e-69630b41da37 tempest-VolumeQuotaClassesTest-1444047577 tempest-VolumeQuotaClassesTest-1444047577-project-admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:24.141709 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-46e43789-97c0-4057-8400-4e16f43850d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0881ebb564a540a1b6bf9f61fd70ea22'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:24.149552 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c3f4d-4523-4bf8-914f-d02a5918b6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:24.150387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c3f4d-4523-4bf8-914f-d02a5918b6b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:24.150528 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:24.150757 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c3f4d-4523-4bf8-914f-d02a5918b6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:24.150945 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c3f4d-4523-4bf8-914f-d02a5918b6b3 admin admin] PATH_INFO: `/v3/users/04da8d850b3a47868513fd82f48937b5` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:24.152251 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c3f4d-4523-4bf8-914f-d02a5918b6b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04da8d850b3a47868513fd82f48937b5)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:24.154303 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c3f4d-4523-4bf8-914f-d02a5918b6b3 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:24.159810 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-46e43789-97c0-4057-8400-4e16f43850d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0881ebb564a540a1b6bf9f61fd70ea22'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:24.162018 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-46e43789-97c0-4057-8400-4e16f43850d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0881ebb564a540a1b6bf9f61fd70ea22'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:24.186864 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1145/2284] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:02:23 2022] DELETE /identity/v3/users/0881ebb564a540a1b6bf9f61fd70ea22 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:24.191818 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5892215-633c-4a4b-83f8-63a53bc5e71e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:24.192105 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5892215-633c-4a4b-83f8-63a53bc5e71e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:24.192386 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5892215-633c-4a4b-83f8-63a53bc5e71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:24.214162 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f5892215-633c-4a4b-83f8-63a53bc5e71e None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:24.214899 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f5892215-633c-4a4b-83f8-63a53bc5e71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:24.216459 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-dd5c3f4d-4523-4bf8-914f-d02a5918b6b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04da8d850b3a47868513fd82f48937b5'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:24.232211 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-dd5c3f4d-4523-4bf8-914f-d02a5918b6b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04da8d850b3a47868513fd82f48937b5'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:24.243830 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-dd5c3f4d-4523-4bf8-914f-d02a5918b6b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04da8d850b3a47868513fd82f48937b5'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:24.247929 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-dd5c3f4d-4523-4bf8-914f-d02a5918b6b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04da8d850b3a47868513fd82f48937b5'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:24.248827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-dd5c3f4d-4523-4bf8-914f-d02a5918b6b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04da8d850b3a47868513fd82f48937b5'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:24.280251 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1140/2285] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:02:24 2022] DELETE /identity/v3/users/04da8d850b3a47868513fd82f48937b5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:24.286386 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e3f1a-10e3-4829-88e6-c21eb576429b None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:24.287028 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e3f1a-10e3-4829-88e6-c21eb576429b None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:24.287917 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e3f1a-10e3-4829-88e6-c21eb576429b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:24.303735 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1146/2286] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:02:24.307408 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5892215-633c-4a4b-83f8-63a53bc5e71e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:24.307852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-676e3f1a-10e3-4829-88e6-c21eb576429b None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:24.309056 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-676e3f1a-10e3-4829-88e6-c21eb576429b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:24.309178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f5892215-633c-4a4b-83f8-63a53bc5e71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:24.389711 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5892215-633c-4a4b-83f8-63a53bc5e71e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:24.400592 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1141/2287] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:02:24.410238 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e3f1a-10e3-4829-88e6-c21eb576429b None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:24.451277 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751992b-2743-4a04-870e-28a2541460c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:24.451999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751992b-2743-4a04-870e-28a2541460c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:24.452233 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751992b-2743-4a04-870e-28a2541460c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:24.452511 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751992b-2743-4a04-870e-28a2541460c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:24.463244 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751992b-2743-4a04-870e-28a2541460c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:24.465313 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751992b-2743-4a04-870e-28a2541460c3 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:24.473933 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:24.476079 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1147/2288] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:02:24.478546 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e3f1a-10e3-4829-88e6-c21eb576429b None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:24.483124 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4d209-4cae-4f36-a978-288bd0ca1460 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:24.484063 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4d209-4cae-4f36-a978-288bd0ca1460 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:24.484540 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4d209-4cae-4f36-a978-288bd0ca1460 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:24.485642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4d209-4cae-4f36-a978-288bd0ca1460 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:24.497780 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b4d209-4cae-4f36-a978-288bd0ca1460 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:24.499544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b4d209-4cae-4f36-a978-288bd0ca1460 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:24.511027 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1142/2289] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:02:24.928100 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751992b-2743-4a04-870e-28a2541460c3 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:24.929704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8751992b-2743-4a04-870e-28a2541460c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:24.943860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751992b-2743-4a04-870e-28a2541460c3 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:24.952047 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db686f-5d5c-4244-abe2-c92c5c8c62c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:24.952616 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db686f-5d5c-4244-abe2-c92c5c8c62c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:24.952995 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db686f-5d5c-4244-abe2-c92c5c8c62c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:24.953339 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db686f-5d5c-4244-abe2-c92c5c8c62c4 admin admin] PATH_INFO: `/v3/projects/57859da276c34a0bb939ceef00f1245c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:24.954845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1db686f-5d5c-4244-abe2-c92c5c8c62c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57859da276c34a0bb939ceef00f1245c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:24.956927 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1db686f-5d5c-4244-abe2-c92c5c8c62c4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:24.963397 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4d209-4cae-4f36-a978-288bd0ca1460 service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:24.964655 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-f7b4d209-4cae-4f36-a978-288bd0ca1460 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:24.995203 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4d209-4cae-4f36-a978-288bd0ca1460 service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:25.004708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6386fe-ec06-45c0-ba77-7f5ab184331f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:25.005609 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6386fe-ec06-45c0-ba77-7f5ab184331f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:25.005984 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6386fe-ec06-45c0-ba77-7f5ab184331f admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:25.006375 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6386fe-ec06-45c0-ba77-7f5ab184331f admin admin] PATH_INFO: `/v3/projects/8fe56fc2f69d43179f5c8175befbe1e0` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:25.007946 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6386fe-ec06-45c0-ba77-7f5ab184331f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fe56fc2f69d43179f5c8175befbe1e0)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:25.010121 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6386fe-ec06-45c0-ba77-7f5ab184331f admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:25.028408 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=1,keystone:DELETE=5,keystone:SELECT=94 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:25.048716 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d1db686f-5d5c-4244-abe2-c92c5c8c62c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57859da276c34a0bb939ceef00f1245c', 'request_id': 'req-d1db686f-5d5c-4244-abe2-c92c5c8c62c4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:25.055571 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d1db686f-5d5c-4244-abe2-c92c5c8c62c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57859da276c34a0bb939ceef00f1245c', 'request_id': 'req-d1db686f-5d5c-4244-abe2-c92c5c8c62c4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:25.063915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d1db686f-5d5c-4244-abe2-c92c5c8c62c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57859da276c34a0bb939ceef00f1245c', 'request_id': 'req-d1db686f-5d5c-4244-abe2-c92c5c8c62c4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:25.065614 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-ca6386fe-ec06-45c0-ba77-7f5ab184331f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fe56fc2f69d43179f5c8175befbe1e0', 'request_id': 'req-ca6386fe-ec06-45c0-ba77-7f5ab184331f'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:25.075695 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-ca6386fe-ec06-45c0-ba77-7f5ab184331f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fe56fc2f69d43179f5c8175befbe1e0', 'request_id': 'req-ca6386fe-ec06-45c0-ba77-7f5ab184331f'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:25.076899 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d1db686f-5d5c-4244-abe2-c92c5c8c62c4 admin admin] The token cache is being invalidate because project 57859da276c34a0bb939ceef00f1245c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:25.077255 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d1db686f-5d5c-4244-abe2-c92c5c8c62c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:25.077953 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d1db686f-5d5c-4244-abe2-c92c5c8c62c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:25.079493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1148/2290] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:24 2022] DELETE /identity/v3/projects/57859da276c34a0bb939ceef00f1245c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:25.082314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-ca6386fe-ec06-45c0-ba77-7f5ab184331f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fe56fc2f69d43179f5c8175befbe1e0', 'request_id': 'req-ca6386fe-ec06-45c0-ba77-7f5ab184331f'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:25.083582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db686f-5d5c-4244-abe2-c92c5c8c62c4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:25.085049 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d1db686f-5d5c-4244-abe2-c92c5c8c62c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:25.096033 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-ca6386fe-ec06-45c0-ba77-7f5ab184331f admin admin] The token cache is being invalidate because project 8fe56fc2f69d43179f5c8175befbe1e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:25.096323 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-ca6386fe-ec06-45c0-ba77-7f5ab184331f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:25.097079 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-ca6386fe-ec06-45c0-ba77-7f5ab184331f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:25.098738 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1143/2291] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:24 2022] DELETE /identity/v3/projects/8fe56fc2f69d43179f5c8175befbe1e0 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:02:25.104116 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3519ed-e313-401c-b50a-e6c33ddc6151 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:25.104474 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3519ed-e313-401c-b50a-e6c33ddc6151 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:25.104759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3519ed-e313-401c-b50a-e6c33ddc6151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:25.126940 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-fc3519ed-e313-401c-b50a-e6c33ddc6151 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:25.128641 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-fc3519ed-e313-401c-b50a-e6c33ddc6151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:25.166845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db686f-5d5c-4244-abe2-c92c5c8c62c4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:25.198672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1144/2292] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:02:25.206722 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405a493-4a21-46d3-8ff3-e212c5a4cc2c None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:25.207031 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405a493-4a21-46d3-8ff3-e212c5a4cc2c None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:25.207341 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405a493-4a21-46d3-8ff3-e212c5a4cc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:25.227169 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b2382-f890-4e37-8fc8-382a0c77a82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:25.228104 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-e405a493-4a21-46d3-8ff3-e212c5a4cc2c None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:25.228410 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b2382-f890-4e37-8fc8-382a0c77a82f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:25.228924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b2382-f890-4e37-8fc8-382a0c77a82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:25.229297 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b2382-f890-4e37-8fc8-382a0c77a82f admin admin] PATH_INFO: `/v3/users/7038898ec63245a8a5f9ae9e0cedf8c7` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:25.229395 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-e405a493-4a21-46d3-8ff3-e212c5a4cc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:25.230660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b2382-f890-4e37-8fc8-382a0c77a82f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7038898ec63245a8a5f9ae9e0cedf8c7)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:25.232788 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b2382-f890-4e37-8fc8-382a0c77a82f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:25.237616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1145/2293] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:02:25.242315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405a493-4a21-46d3-8ff3-e212c5a4cc2c None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:25.252036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405a493-4a21-46d3-8ff3-e212c5a4cc2c None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:25.257710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d5bda6-3056-4437-ad9e-d1129b4c873f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:25.258433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5bda6-3056-4437-ad9e-d1129b4c873f admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:25.258713 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5bda6-3056-4437-ad9e-d1129b4c873f admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:25.258990 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5bda6-3056-4437-ad9e-d1129b4c873f admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:25.259752 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d5bda6-3056-4437-ad9e-d1129b4c873f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:02:25.260057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d5bda6-3056-4437-ad9e-d1129b4c873f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:25.261704 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d5bda6-3056-4437-ad9e-d1129b4c873f admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:25.275602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1146/2294] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 05:02:25.280079 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d5bda6-3056-4437-ad9e-d1129b4c873f admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:25.292256 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d5bda6-3056-4437-ad9e-d1129b4c873f admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:25.297341 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00412fd7-29be-4dd7-b8ec-c6766f1c4d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:25.297635 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-eb3b2382-f890-4e37-8fc8-382a0c77a82f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7038898ec63245a8a5f9ae9e0cedf8c7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:25.297975 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00412fd7-29be-4dd7-b8ec-c6766f1c4d26 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:25.298258 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00412fd7-29be-4dd7-b8ec-c6766f1c4d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:25.298534 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00412fd7-29be-4dd7-b8ec-c6766f1c4d26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:25.299244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00412fd7-29be-4dd7-b8ec-c6766f1c4d26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:25.301276 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00412fd7-29be-4dd7-b8ec-c6766f1c4d26 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:25.307888 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-eb3b2382-f890-4e37-8fc8-382a0c77a82f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7038898ec63245a8a5f9ae9e0cedf8c7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:25.308633 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-eb3b2382-f890-4e37-8fc8-382a0c77a82f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7038898ec63245a8a5f9ae9e0cedf8c7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:25.332158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-eb3b2382-f890-4e37-8fc8-382a0c77a82f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7038898ec63245a8a5f9ae9e0cedf8c7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:25.333044 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-eb3b2382-f890-4e37-8fc8-382a0c77a82f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7038898ec63245a8a5f9ae9e0cedf8c7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:25.336098 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:25.349137 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1147/2295] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:02:25 2022] POST /identity/v3/projects => generated 365 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:25.355868 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00412fd7-29be-4dd7-b8ec-c6766f1c4d26 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:25.361269 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-00412fd7-29be-4dd7-b8ec-c6766f1c4d26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:25.367663 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1149/2296] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:02:25 2022] DELETE /identity/v3/users/7038898ec63245a8a5f9ae9e0cedf8c7 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:02:25.413002 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:25.454414 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00412fd7-29be-4dd7-b8ec-c6766f1c4d26 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:25.539096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa83e34-c378-4b82-8280-c9568339f4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:25.540274 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa83e34-c378-4b82-8280-c9568339f4f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:25.540769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa83e34-c378-4b82-8280-c9568339f4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:25.541325 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa83e34-c378-4b82-8280-c9568339f4f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:25.542172 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa83e34-c378-4b82-8280-c9568339f4f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:25.544490 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa83e34-c378-4b82-8280-c9568339f4f5 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:25.569268 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1148/2297] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:02:25 2022] POST /identity/v3/users => generated 283 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:02:25.574127 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b2382-f890-4e37-8fc8-382a0c77a82f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:25.585327 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b2382-f890-4e37-8fc8-382a0c77a82f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:25.590241 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3f89f-89dc-4abe-badf-d47b7127f5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:25.591017 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3f89f-89dc-4abe-badf-d47b7127f5ce admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:25.591434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3f89f-89dc-4abe-badf-d47b7127f5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:25.591831 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3f89f-89dc-4abe-badf-d47b7127f5ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:25.592880 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b3f89f-89dc-4abe-badf-d47b7127f5ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:25.594476 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b3f89f-89dc-4abe-badf-d47b7127f5ce admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:25.610046 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1150/2298] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:25.611675 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa83e34-c378-4b82-8280-c9568339f4f5 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:25.623817 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa83e34-c378-4b82-8280-c9568339f4f5 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:25.632188 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93a252-3188-493e-ae53-3579cb923176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:25.633181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93a252-3188-493e-ae53-3579cb923176 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:25.633181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93a252-3188-493e-ae53-3579cb923176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:25.633567 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93a252-3188-493e-ae53-3579cb923176 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:25.634514 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f93a252-3188-493e-ae53-3579cb923176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:25.637121 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f93a252-3188-493e-ae53-3579cb923176 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:25.650367 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1149/2299] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:25.654763 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3f89f-89dc-4abe-badf-d47b7127f5ce admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:25.666979 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3f89f-89dc-4abe-badf-d47b7127f5ce admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:25.673295 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9813628b-3f9f-4d5d-b865-b372d04016f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:25.676816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9813628b-3f9f-4d5d-b865-b372d04016f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:25.677104 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9813628b-3f9f-4d5d-b865-b372d04016f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:25.677380 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9813628b-3f9f-4d5d-b865-b372d04016f4 admin admin] PATH_INFO: `/v3/projects/a049ed8c11ae45ae9ed14757f1e00b00/users/fa8e1ff7b31d47b29c7fe11b0eb9dce4/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:25.698773 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9813628b-3f9f-4d5d-b865-b372d04016f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a049ed8c11ae45ae9ed14757f1e00b00, user_id=fa8e1ff7b31d47b29c7fe11b0eb9dce4, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:25.701564 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9813628b-3f9f-4d5d-b865-b372d04016f4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:25.720322 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1151/2300] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:25 2022] PUT /identity/v3/projects/a049ed8c11ae45ae9ed14757f1e00b00/users/fa8e1ff7b31d47b29c7fe11b0eb9dce4/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:25.731007 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae654fcc-3aba-4546-b13a-a8f555d4ce57 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:25.731378 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae654fcc-3aba-4546-b13a-a8f555d4ce57 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:25.731609 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae654fcc-3aba-4546-b13a-a8f555d4ce57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:25.767992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-ae654fcc-3aba-4546-b13a-a8f555d4ce57 None None] MFA Rules not processed for user `fa8e1ff7b31d47b29c7fe11b0eb9dce4`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:25.770405 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-ae654fcc-3aba-4546-b13a-a8f555d4ce57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:25.845965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1150/2301] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:25 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:25.855156 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9813628b-3f9f-4d5d-b865-b372d04016f4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:25.856818 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9813628b-3f9f-4d5d-b865-b372d04016f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:25.860847 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae654fcc-3aba-4546-b13a-a8f555d4ce57 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:25.869193 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-ae654fcc-3aba-4546-b13a-a8f555d4ce57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:25.927410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:25.931733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae654fcc-3aba-4546-b13a-a8f555d4ce57 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:25.936252 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1f0f6-6ac7-4b74-b019-bd17d4b7efb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:25.939078 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1f0f6-6ac7-4b74-b019-bd17d4b7efb5 service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:25.939078 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1f0f6-6ac7-4b74-b019-bd17d4b7efb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:25.939078 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1f0f6-6ac7-4b74-b019-bd17d4b7efb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:25.939430 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9813628b-3f9f-4d5d-b865-b372d04016f4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:25.943395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55710fb7-a0b0-4954-a6b4-6e05c89cc3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:25.944222 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55710fb7-a0b0-4954-a6b4-6e05c89cc3bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:25.944581 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55710fb7-a0b0-4954-a6b4-6e05c89cc3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:25.944915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55710fb7-a0b0-4954-a6b4-6e05c89cc3bf admin admin] PATH_INFO: `/v3/projects/c9171fb681ec49dcbb86ed9aca419650` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:25.946164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55710fb7-a0b0-4954-a6b4-6e05c89cc3bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9171fb681ec49dcbb86ed9aca419650)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:25.948555 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55710fb7-a0b0-4954-a6b4-6e05c89cc3bf admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:25.948674 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a1f0f6-6ac7-4b74-b019-bd17d4b7efb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:25.950247 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a1f0f6-6ac7-4b74-b019-bd17d4b7efb5 service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:25.961020 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1151/2302] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:25 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:02:26.002930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-55710fb7-a0b0-4954-a6b4-6e05c89cc3bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9171fb681ec49dcbb86ed9aca419650', 'request_id': 'req-55710fb7-a0b0-4954-a6b4-6e05c89cc3bf'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:26.010029 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-55710fb7-a0b0-4954-a6b4-6e05c89cc3bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9171fb681ec49dcbb86ed9aca419650', 'request_id': 'req-55710fb7-a0b0-4954-a6b4-6e05c89cc3bf'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:26.022640 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-55710fb7-a0b0-4954-a6b4-6e05c89cc3bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9171fb681ec49dcbb86ed9aca419650', 'request_id': 'req-55710fb7-a0b0-4954-a6b4-6e05c89cc3bf'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:26.036411 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-55710fb7-a0b0-4954-a6b4-6e05c89cc3bf admin admin] The token cache is being invalidate because project c9171fb681ec49dcbb86ed9aca419650 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:26.036915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-55710fb7-a0b0-4954-a6b4-6e05c89cc3bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:26.037769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-55710fb7-a0b0-4954-a6b4-6e05c89cc3bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:26.042936 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1152/2303] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:25 2022] DELETE /identity/v3/projects/c9171fb681ec49dcbb86ed9aca419650 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:26.044631 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0d2e2-ef25-47ab-a12e-d49824dce58a None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:26.044916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0d2e2-ef25-47ab-a12e-d49824dce58a None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:26.045182 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0d2e2-ef25-47ab-a12e-d49824dce58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:26.070000 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-51f0d2e2-ef25-47ab-a12e-d49824dce58a None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:26.071231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-51f0d2e2-ef25-47ab-a12e-d49824dce58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:26.149302 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1152/2304] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:26.158815 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a5f84-5c45-4bb4-ba59-0fc240a20049 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:26.159815 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a5f84-5c45-4bb4-ba59-0fc240a20049 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:26.160295 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a5f84-5c45-4bb4-ba59-0fc240a20049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:26.183038 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-af8a5f84-5c45-4bb4-ba59-0fc240a20049 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:26.184275 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-af8a5f84-5c45-4bb4-ba59-0fc240a20049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:26.192461 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1153/2305] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:26.197065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0d2e2-ef25-47ab-a12e-d49824dce58a None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:26.209181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0d2e2-ef25-47ab-a12e-d49824dce58a None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:26.214371 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2731d3-9053-4564-896e-7ba34f1eecf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:26.215125 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2731d3-9053-4564-896e-7ba34f1eecf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:26.215474 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2731d3-9053-4564-896e-7ba34f1eecf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:26.215833 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2731d3-9053-4564-896e-7ba34f1eecf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:26.216603 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2731d3-9053-4564-896e-7ba34f1eecf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:02:26.216968 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2731d3-9053-4564-896e-7ba34f1eecf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:26.218907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2731d3-9053-4564-896e-7ba34f1eecf1 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:26.250369 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1153/2306] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:02:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:02:26.251715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a5f84-5c45-4bb4-ba59-0fc240a20049 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:26.264285 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a5f84-5c45-4bb4-ba59-0fc240a20049 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:26.270264 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b2f29e-cbad-4260-a781-4d6fc1fe2df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:26.271016 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2f29e-cbad-4260-a781-4d6fc1fe2df3 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:26.271435 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2f29e-cbad-4260-a781-4d6fc1fe2df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:26.272241 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2f29e-cbad-4260-a781-4d6fc1fe2df3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:26.273108 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b2f29e-cbad-4260-a781-4d6fc1fe2df3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:26.276968 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b2f29e-cbad-4260-a781-4d6fc1fe2df3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:26.318903 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1154/2307] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:02:26 2022] POST /identity/v3/projects => generated 407 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:26.320439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2731d3-9053-4564-896e-7ba34f1eecf1 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:26.334573 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2731d3-9053-4564-896e-7ba34f1eecf1 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:26.402056 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90435c40-4a15-417c-8bae-673dd0a5984c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:26.402597 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90435c40-4a15-417c-8bae-673dd0a5984c admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:26.402877 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90435c40-4a15-417c-8bae-673dd0a5984c admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:26.403147 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90435c40-4a15-417c-8bae-673dd0a5984c admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:26.403869 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90435c40-4a15-417c-8bae-673dd0a5984c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:26.405807 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90435c40-4a15-417c-8bae-673dd0a5984c admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:26.430877 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1154/2308] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:02:26 2022] POST /identity/v3/users => generated 304 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:26.435334 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b2f29e-cbad-4260-a781-4d6fc1fe2df3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:26.446181 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b2f29e-cbad-4260-a781-4d6fc1fe2df3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:26.451713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642774bb-9c20-4aa4-9763-65c1fd366513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:26.452337 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642774bb-9c20-4aa4-9763-65c1fd366513 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:26.452653 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642774bb-9c20-4aa4-9763-65c1fd366513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:26.452895 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642774bb-9c20-4aa4-9763-65c1fd366513 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:26.453592 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642774bb-9c20-4aa4-9763-65c1fd366513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:26.455193 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642774bb-9c20-4aa4-9763-65c1fd366513 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:26.457583 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:26.467008 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1155/2309] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:26.471683 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90435c40-4a15-417c-8bae-673dd0a5984c admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:26.482194 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90435c40-4a15-417c-8bae-673dd0a5984c admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:26.487412 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb2ca0-20b8-486a-a1f5-bfe326bcdd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:26.488080 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb2ca0-20b8-486a-a1f5-bfe326bcdd31 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:26.488365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb2ca0-20b8-486a-a1f5-bfe326bcdd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:26.488641 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb2ca0-20b8-486a-a1f5-bfe326bcdd31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:26.489353 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb2ca0-20b8-486a-a1f5-bfe326bcdd31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:26.491037 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb2ca0-20b8-486a-a1f5-bfe326bcdd31 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:26.501672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1155/2310] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:26.506392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642774bb-9c20-4aa4-9763-65c1fd366513 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:26.517550 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642774bb-9c20-4aa4-9763-65c1fd366513 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:26.524650 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be332045-171b-4e3e-a25b-bc4a32612c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:26.525362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be332045-171b-4e3e-a25b-bc4a32612c32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:26.525636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be332045-171b-4e3e-a25b-bc4a32612c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:26.525915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be332045-171b-4e3e-a25b-bc4a32612c32 admin admin] PATH_INFO: `/v3/projects/30fdaa280cca4b43a0e81ae08d0dbf04/users/0e8389f747fd4c0e9f9d0d41f13f975c/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:26.548946 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be332045-171b-4e3e-a25b-bc4a32612c32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30fdaa280cca4b43a0e81ae08d0dbf04, user_id=0e8389f747fd4c0e9f9d0d41f13f975c, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:26.552629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be332045-171b-4e3e-a25b-bc4a32612c32 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:26.567845 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1156/2311] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:26 2022] PUT /identity/v3/projects/30fdaa280cca4b43a0e81ae08d0dbf04/users/0e8389f747fd4c0e9f9d0d41f13f975c/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:26.578990 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ac7f1-3c4e-4f3f-aab5-657a36755909 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:26.579364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ac7f1-3c4e-4f3f-aab5-657a36755909 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:26.579715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ac7f1-3c4e-4f3f-aab5-657a36755909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:26.610191 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-502ac7f1-3c4e-4f3f-aab5-657a36755909 None None] MFA Rules not processed for user `0e8389f747fd4c0e9f9d0d41f13f975c`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:26.611469 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-502ac7f1-3c4e-4f3f-aab5-657a36755909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:26.665292 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:26.689570 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1156/2312] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:26 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:26.694383 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be332045-171b-4e3e-a25b-bc4a32612c32 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:26.765750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be332045-171b-4e3e-a25b-bc4a32612c32 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:26.770800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d93e4-31fc-4d04-b626-32ba1943e106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:26.771460 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d93e4-31fc-4d04-b626-32ba1943e106 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:26.771881 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d93e4-31fc-4d04-b626-32ba1943e106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:26.772289 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d93e4-31fc-4d04-b626-32ba1943e106 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:26.772954 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d93e4-31fc-4d04-b626-32ba1943e106 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:26.774932 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d93e4-31fc-4d04-b626-32ba1943e106 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:26.807127 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1157/2313] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:02:26 2022] POST /identity/v3/projects => generated 407 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:26.817183 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ac7f1-3c4e-4f3f-aab5-657a36755909 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:26.829064 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ac7f1-3c4e-4f3f-aab5-657a36755909 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:26.886564 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f98ed-8d76-491b-b8b8-85e813d80108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:26.887359 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f98ed-8d76-491b-b8b8-85e813d80108 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:26.887676 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f98ed-8d76-491b-b8b8-85e813d80108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:26.887956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f98ed-8d76-491b-b8b8-85e813d80108 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:26.888669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f98ed-8d76-491b-b8b8-85e813d80108 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:26.890664 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f98ed-8d76-491b-b8b8-85e813d80108 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:26.915233 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1157/2314] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:02:26 2022] POST /identity/v3/users => generated 310 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:26.919286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d93e4-31fc-4d04-b626-32ba1943e106 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:26.929976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d93e4-31fc-4d04-b626-32ba1943e106 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:26.934606 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d06561-e3c0-44a9-b908-c6462bbc110a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:26.935230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d06561-e3c0-44a9-b908-c6462bbc110a admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:26.935511 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d06561-e3c0-44a9-b908-c6462bbc110a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:26.935816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d06561-e3c0-44a9-b908-c6462bbc110a admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:26.936516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d06561-e3c0-44a9-b908-c6462bbc110a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:26.938150 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d06561-e3c0-44a9-b908-c6462bbc110a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:26.949262 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1158/2315] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:26.953709 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f98ed-8d76-491b-b8b8-85e813d80108 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:26.968417 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f98ed-8d76-491b-b8b8-85e813d80108 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:26.977368 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd4cc8-a7f2-401a-a4e8-ed8323dbc7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:26.978068 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd4cc8-a7f2-401a-a4e8-ed8323dbc7ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:26.978351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd4cc8-a7f2-401a-a4e8-ed8323dbc7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:26.978624 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd4cc8-a7f2-401a-a4e8-ed8323dbc7ec admin admin] PATH_INFO: `/v3/projects/e2373880d78045f7a025d3f70ebb22ad/users/366c068eaf9b4203a3f3abcb1f612b11/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:26.995269 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd4cc8-a7f2-401a-a4e8-ed8323dbc7ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2373880d78045f7a025d3f70ebb22ad, user_id=366c068eaf9b4203a3f3abcb1f612b11, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:26.998036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd4cc8-a7f2-401a-a4e8-ed8323dbc7ec admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:27.014468 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1158/2316] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:26 2022] PUT /identity/v3/projects/e2373880d78045f7a025d3f70ebb22ad/users/366c068eaf9b4203a3f3abcb1f612b11/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:02:27.018779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d06561-e3c0-44a9-b908-c6462bbc110a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:27.029774 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d06561-e3c0-44a9-b908-c6462bbc110a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:27.089752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252d626-b591-42a7-96e6-1fb8610a16b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:27.090402 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252d626-b591-42a7-96e6-1fb8610a16b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:27.090702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252d626-b591-42a7-96e6-1fb8610a16b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:27.090970 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252d626-b591-42a7-96e6-1fb8610a16b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:27.091701 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d252d626-b591-42a7-96e6-1fb8610a16b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:27.093298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d252d626-b591-42a7-96e6-1fb8610a16b6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:27.107577 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1159/2317] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:27.108794 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd4cc8-a7f2-401a-a4e8-ed8323dbc7ec admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:27.119425 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd4cc8-a7f2-401a-a4e8-ed8323dbc7ec admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:27.124489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02360e68-cb73-4628-83cd-7601a9815b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:27.125275 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02360e68-cb73-4628-83cd-7601a9815b2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:27.125579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02360e68-cb73-4628-83cd-7601a9815b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:27.125819 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02360e68-cb73-4628-83cd-7601a9815b2b admin admin] PATH_INFO: `/v3/projects/e2373880d78045f7a025d3f70ebb22ad/users/366c068eaf9b4203a3f3abcb1f612b11/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:27.128061 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02360e68-cb73-4628-83cd-7601a9815b2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2373880d78045f7a025d3f70ebb22ad, user_id=366c068eaf9b4203a3f3abcb1f612b11, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:27.130907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02360e68-cb73-4628-83cd-7601a9815b2b admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:27.141604 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1159/2318] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:27 2022] PUT /identity/v3/projects/e2373880d78045f7a025d3f70ebb22ad/users/366c068eaf9b4203a3f3abcb1f612b11/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:27.152040 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb6a29-15b4-41a4-b3c6-0517cbc814e6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:27.152339 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb6a29-15b4-41a4-b3c6-0517cbc814e6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:27.152622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb6a29-15b4-41a4-b3c6-0517cbc814e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:27.181805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6beb6a29-15b4-41a4-b3c6-0517cbc814e6 None None] MFA Rules not processed for user `366c068eaf9b4203a3f3abcb1f612b11`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:27.182928 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6beb6a29-15b4-41a4-b3c6-0517cbc814e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:27.218082 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:27.258597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1160/2319] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:27 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:02:27.273341 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02360e68-cb73-4628-83cd-7601a9815b2b admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:27.276113 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-02360e68-cb73-4628-83cd-7601a9815b2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:27.356085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02360e68-cb73-4628-83cd-7601a9815b2b admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:27.360024 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e495ee-a358-4ebe-b484-0e4ed4234cf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:27.360675 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e495ee-a358-4ebe-b484-0e4ed4234cf0 service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:27.360960 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e495ee-a358-4ebe-b484-0e4ed4234cf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:27.361301 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e495ee-a358-4ebe-b484-0e4ed4234cf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:27.372507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e495ee-a358-4ebe-b484-0e4ed4234cf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:27.373995 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e495ee-a358-4ebe-b484-0e4ed4234cf0 service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:27.384455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1160/2320] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:27 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:02:27.558277 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb6a29-15b4-41a4-b3c6-0517cbc814e6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:27.569516 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb6a29-15b4-41a4-b3c6-0517cbc814e6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:27.573563 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a327c-465f-4854-997e-986d2db41908 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:27.574192 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a327c-465f-4854-997e-986d2db41908 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:27.574473 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a327c-465f-4854-997e-986d2db41908 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:27.574746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a327c-465f-4854-997e-986d2db41908 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:27.639120 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a327c-465f-4854-997e-986d2db41908 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:27.640657 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a327c-465f-4854-997e-986d2db41908 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:27.650784 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1161/2321] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:27 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:02:37.379350 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=76,keystone:INSERT=5 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:37.645897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=42 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:38.330123 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e495ee-a358-4ebe-b484-0e4ed4234cf0 service cinder] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:38.402306 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e495ee-a358-4ebe-b484-0e4ed4234cf0 service cinder] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:38.407378 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd9a44-23b2-4d06-a464-58834fcad1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:38.408101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd9a44-23b2-4d06-a464-58834fcad1ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:38.408496 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd9a44-23b2-4d06-a464-58834fcad1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:38.408749 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd9a44-23b2-4d06-a464-58834fcad1ad admin admin] PATH_INFO: `/v3/users/0e8389f747fd4c0e9f9d0d41f13f975c` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:38.410017 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd9a44-23b2-4d06-a464-58834fcad1ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e8389f747fd4c0e9f9d0d41f13f975c)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:38.412372 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd9a44-23b2-4d06-a464-58834fcad1ad admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:38.465239 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-a4bd9a44-23b2-4d06-a464-58834fcad1ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e8389f747fd4c0e9f9d0d41f13f975c'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:38.473415 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-a4bd9a44-23b2-4d06-a464-58834fcad1ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e8389f747fd4c0e9f9d0d41f13f975c'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:38.484361 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-a4bd9a44-23b2-4d06-a464-58834fcad1ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e8389f747fd4c0e9f9d0d41f13f975c'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:38.485041 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-a4bd9a44-23b2-4d06-a464-58834fcad1ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e8389f747fd4c0e9f9d0d41f13f975c'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:38.485668 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-a4bd9a44-23b2-4d06-a464-58834fcad1ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e8389f747fd4c0e9f9d0d41f13f975c'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:38.506184 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1161/2322] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:02:38 2022] DELETE /identity/v3/users/0e8389f747fd4c0e9f9d0d41f13f975c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:02:38.511196 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f53ff-b3ae-4eef-8a6d-4a6083b04e5a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:38.511496 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f53ff-b3ae-4eef-8a6d-4a6083b04e5a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:38.511846 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f53ff-b3ae-4eef-8a6d-4a6083b04e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:38.533722 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d32f53ff-b3ae-4eef-8a6d-4a6083b04e5a None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:38.534983 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d32f53ff-b3ae-4eef-8a6d-4a6083b04e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:38.604378 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1162/2323] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:38.616125 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd9a44-23b2-4d06-a464-58834fcad1ad admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:38.617945 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a4bd9a44-23b2-4d06-a464-58834fcad1ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:38.677665 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd9a44-23b2-4d06-a464-58834fcad1ad admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:38.681510 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924584b3-cbce-4279-aba0-757b300aaad8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:38.682207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924584b3-cbce-4279-aba0-757b300aaad8 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:38.682431 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924584b3-cbce-4279-aba0-757b300aaad8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:38.682701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924584b3-cbce-4279-aba0-757b300aaad8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:38.693121 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924584b3-cbce-4279-aba0-757b300aaad8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:38.695194 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924584b3-cbce-4279-aba0-757b300aaad8 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:38.707669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1162/2324] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:02:38.992440 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f53ff-b3ae-4eef-8a6d-4a6083b04e5a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:38.994033 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d32f53ff-b3ae-4eef-8a6d-4a6083b04e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:39.006586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f53ff-b3ae-4eef-8a6d-4a6083b04e5a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:39.011370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f4ec5-47e8-4194-b2b5-6f5e7f9ef7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:39.012066 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f4ec5-47e8-4194-b2b5-6f5e7f9ef7a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:39.012353 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f4ec5-47e8-4194-b2b5-6f5e7f9ef7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:39.012631 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f4ec5-47e8-4194-b2b5-6f5e7f9ef7a5 admin admin] PATH_INFO: `/v3/projects/30fdaa280cca4b43a0e81ae08d0dbf04` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:39.013782 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f4ec5-47e8-4194-b2b5-6f5e7f9ef7a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30fdaa280cca4b43a0e81ae08d0dbf04)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:39.015762 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f4ec5-47e8-4194-b2b5-6f5e7f9ef7a5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:39.065231 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2f5f4ec5-47e8-4194-b2b5-6f5e7f9ef7a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30fdaa280cca4b43a0e81ae08d0dbf04', 'request_id': 'req-2f5f4ec5-47e8-4194-b2b5-6f5e7f9ef7a5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:39.072352 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2f5f4ec5-47e8-4194-b2b5-6f5e7f9ef7a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30fdaa280cca4b43a0e81ae08d0dbf04', 'request_id': 'req-2f5f4ec5-47e8-4194-b2b5-6f5e7f9ef7a5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:39.081577 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2f5f4ec5-47e8-4194-b2b5-6f5e7f9ef7a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30fdaa280cca4b43a0e81ae08d0dbf04', 'request_id': 'req-2f5f4ec5-47e8-4194-b2b5-6f5e7f9ef7a5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:39.094297 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2f5f4ec5-47e8-4194-b2b5-6f5e7f9ef7a5 admin admin] The token cache is being invalidate because project 30fdaa280cca4b43a0e81ae08d0dbf04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:39.094585 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2f5f4ec5-47e8-4194-b2b5-6f5e7f9ef7a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:39.095305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2f5f4ec5-47e8-4194-b2b5-6f5e7f9ef7a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:39.097378 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1163/2325] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:38 2022] DELETE /identity/v3/projects/30fdaa280cca4b43a0e81ae08d0dbf04 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:39.101853 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924584b3-cbce-4279-aba0-757b300aaad8 service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:39.112221 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924584b3-cbce-4279-aba0-757b300aaad8 service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:39.134885 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:39.178850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d2caf-1ed3-4cc4-a3a8-f81e6ffab52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:39.179524 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d2caf-1ed3-4cc4-a3a8-f81e6ffab52c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:39.179842 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d2caf-1ed3-4cc4-a3a8-f81e6ffab52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:39.180179 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d2caf-1ed3-4cc4-a3a8-f81e6ffab52c admin admin] PATH_INFO: `/v3/users/366c068eaf9b4203a3f3abcb1f612b11` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:39.181350 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8d2caf-1ed3-4cc4-a3a8-f81e6ffab52c admin admin] RBAC: Authorizing `identity:delete_user(user_id=366c068eaf9b4203a3f3abcb1f612b11)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:39.183385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8d2caf-1ed3-4cc4-a3a8-f81e6ffab52c admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:39.266177 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8a8d2caf-1ed3-4cc4-a3a8-f81e6ffab52c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '366c068eaf9b4203a3f3abcb1f612b11'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:39.277274 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8a8d2caf-1ed3-4cc4-a3a8-f81e6ffab52c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '366c068eaf9b4203a3f3abcb1f612b11'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:39.294883 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8a8d2caf-1ed3-4cc4-a3a8-f81e6ffab52c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '366c068eaf9b4203a3f3abcb1f612b11'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:39.296324 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8a8d2caf-1ed3-4cc4-a3a8-f81e6ffab52c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '366c068eaf9b4203a3f3abcb1f612b11'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:39.297102 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8a8d2caf-1ed3-4cc4-a3a8-f81e6ffab52c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '366c068eaf9b4203a3f3abcb1f612b11'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:39.322508 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1163/2326] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:02:39 2022] DELETE /identity/v3/users/366c068eaf9b4203a3f3abcb1f612b11 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:39.746236 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f4ec5-47e8-4194-b2b5-6f5e7f9ef7a5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:39.747798 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2f5f4ec5-47e8-4194-b2b5-6f5e7f9ef7a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:39.761125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f4ec5-47e8-4194-b2b5-6f5e7f9ef7a5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:39.822811 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56843071-a70b-4440-a55e-4ae63a32b900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:39.823254 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56843071-a70b-4440-a55e-4ae63a32b900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:39.823515 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56843071-a70b-4440-a55e-4ae63a32b900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:39.823826 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56843071-a70b-4440-a55e-4ae63a32b900 admin admin] PATH_INFO: `/v3/projects/e2373880d78045f7a025d3f70ebb22ad` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:39.825147 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56843071-a70b-4440-a55e-4ae63a32b900 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2373880d78045f7a025d3f70ebb22ad)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:39.827241 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56843071-a70b-4440-a55e-4ae63a32b900 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:39.827784 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d2caf-1ed3-4cc4-a3a8-f81e6ffab52c admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:39.829338 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-8a8d2caf-1ed3-4cc4-a3a8-f81e6ffab52c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:39.843438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d2caf-1ed3-4cc4-a3a8-f81e6ffab52c admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:39.848318 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da831b-348c-4084-a27d-b02c9ea7c565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:39.848964 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da831b-348c-4084-a27d-b02c9ea7c565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:39.849235 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da831b-348c-4084-a27d-b02c9ea7c565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:39.849531 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da831b-348c-4084-a27d-b02c9ea7c565 admin admin] PATH_INFO: `/v3/users/b9a38fcd74ca4a358ec6f580d0351b28` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:39.850731 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da831b-348c-4084-a27d-b02c9ea7c565 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9a38fcd74ca4a358ec6f580d0351b28)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:39.852751 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da831b-348c-4084-a27d-b02c9ea7c565 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:39.861425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:39.880733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-56843071-a70b-4440-a55e-4ae63a32b900 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2373880d78045f7a025d3f70ebb22ad', 'request_id': 'req-56843071-a70b-4440-a55e-4ae63a32b900'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:39.888381 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-56843071-a70b-4440-a55e-4ae63a32b900 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2373880d78045f7a025d3f70ebb22ad', 'request_id': 'req-56843071-a70b-4440-a55e-4ae63a32b900'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:39.896924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-56843071-a70b-4440-a55e-4ae63a32b900 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2373880d78045f7a025d3f70ebb22ad', 'request_id': 'req-56843071-a70b-4440-a55e-4ae63a32b900'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:39.905330 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-19da831b-348c-4084-a27d-b02c9ea7c565 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9a38fcd74ca4a358ec6f580d0351b28'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:39.908386 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-56843071-a70b-4440-a55e-4ae63a32b900 admin admin] The token cache is being invalidate because project e2373880d78045f7a025d3f70ebb22ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:39.908678 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-56843071-a70b-4440-a55e-4ae63a32b900 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:39.909327 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-56843071-a70b-4440-a55e-4ae63a32b900 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:39.911094 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1164/2327] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:39 2022] DELETE /identity/v3/projects/e2373880d78045f7a025d3f70ebb22ad => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:39.912730 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-19da831b-348c-4084-a27d-b02c9ea7c565 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9a38fcd74ca4a358ec6f580d0351b28'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:39.915662 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52121926-8b57-4eea-9d6d-f03213dd8ee5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:39.915950 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52121926-8b57-4eea-9d6d-f03213dd8ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:39.916331 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52121926-8b57-4eea-9d6d-f03213dd8ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:39.920801 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-19da831b-348c-4084-a27d-b02c9ea7c565 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9a38fcd74ca4a358ec6f580d0351b28'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:39.921548 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-19da831b-348c-4084-a27d-b02c9ea7c565 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9a38fcd74ca4a358ec6f580d0351b28'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:39.922171 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-19da831b-348c-4084-a27d-b02c9ea7c565 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9a38fcd74ca4a358ec6f580d0351b28'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:39.936206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-52121926-8b57-4eea-9d6d-f03213dd8ee5 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:39.937414 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-52121926-8b57-4eea-9d6d-f03213dd8ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:39.939783 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1164/2328] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:02:39 2022] DELETE /identity/v3/users/b9a38fcd74ca4a358ec6f580d0351b28 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:39.945006 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a3ecb-4b6e-4f14-862f-c42539f02905 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:39.945306 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a3ecb-4b6e-4f14-862f-c42539f02905 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:39.945575 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a3ecb-4b6e-4f14-862f-c42539f02905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:39.966180 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-e31a3ecb-4b6e-4f14-862f-c42539f02905 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:39.967364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-e31a3ecb-4b6e-4f14-862f-c42539f02905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:39.976439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:39.999261 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1165/2329] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:02:40.008301 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979ecd6-2f55-4038-aca4-e06b57307392 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:40.008720 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979ecd6-2f55-4038-aca4-e06b57307392 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:40.008996 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979ecd6-2f55-4038-aca4-e06b57307392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:40.029053 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c979ecd6-2f55-4038-aca4-e06b57307392 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:40.030297 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c979ecd6-2f55-4038-aca4-e06b57307392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:40.030675 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1165/2330] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:40.038843 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1166/2331] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:02:40.040922 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a3ecb-4b6e-4f14-862f-c42539f02905 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:40.042374 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-e31a3ecb-4b6e-4f14-862f-c42539f02905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:40.042992 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979ecd6-2f55-4038-aca4-e06b57307392 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:40.053340 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979ecd6-2f55-4038-aca4-e06b57307392 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:40.059541 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f030b-0da9-4fc5-b2ad-adc9c8198208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:40.060188 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f030b-0da9-4fc5-b2ad-adc9c8198208 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:40.060474 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f030b-0da9-4fc5-b2ad-adc9c8198208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:40.060754 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f030b-0da9-4fc5-b2ad-adc9c8198208 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:40.061493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f030b-0da9-4fc5-b2ad-adc9c8198208 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:02:40.061804 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f030b-0da9-4fc5-b2ad-adc9c8198208 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:40.063420 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f030b-0da9-4fc5-b2ad-adc9c8198208 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:40.076249 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1167/2332] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 05:02:40.082326 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f030b-0da9-4fc5-b2ad-adc9c8198208 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:40.093395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f030b-0da9-4fc5-b2ad-adc9c8198208 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:40.098318 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef2e09-d78f-4df1-aa5e-a8df87508a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:40.098999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef2e09-d78f-4df1-aa5e-a8df87508a63 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:40.099233 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef2e09-d78f-4df1-aa5e-a8df87508a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:40.099511 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef2e09-d78f-4df1-aa5e-a8df87508a63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:40.100208 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eef2e09-d78f-4df1-aa5e-a8df87508a63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:40.102255 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eef2e09-d78f-4df1-aa5e-a8df87508a63 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:40.107283 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a3ecb-4b6e-4f14-862f-c42539f02905 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:40.111474 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042fc834-3eee-4d9c-9233-ba522437d33d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:40.112261 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042fc834-3eee-4d9c-9233-ba522437d33d service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:40.112544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042fc834-3eee-4d9c-9233-ba522437d33d service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:40.112877 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042fc834-3eee-4d9c-9233-ba522437d33d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:40.123530 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042fc834-3eee-4d9c-9233-ba522437d33d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:40.125292 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042fc834-3eee-4d9c-9233-ba522437d33d service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:40.135852 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1168/2333] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:02:40 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:40.139578 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1166/2334] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:02:40.140453 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef2e09-d78f-4df1-aa5e-a8df87508a63 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:40.210419 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef2e09-d78f-4df1-aa5e-a8df87508a63 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:40.216426 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16caa5f-8b38-40a2-b5b8-612460406569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:40.217000 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16caa5f-8b38-40a2-b5b8-612460406569 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:40.217321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16caa5f-8b38-40a2-b5b8-612460406569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:40.217552 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16caa5f-8b38-40a2-b5b8-612460406569 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:40.218217 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16caa5f-8b38-40a2-b5b8-612460406569 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:40.220397 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16caa5f-8b38-40a2-b5b8-612460406569 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:40.244851 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1169/2335] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:02:40 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:40.249758 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042fc834-3eee-4d9c-9233-ba522437d33d service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:40.259563 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042fc834-3eee-4d9c-9233-ba522437d33d service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:40.264831 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3294ba-f16e-4df6-bc5e-3151299ec91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:40.265455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3294ba-f16e-4df6-bc5e-3151299ec91f admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:40.265739 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3294ba-f16e-4df6-bc5e-3151299ec91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:40.266010 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3294ba-f16e-4df6-bc5e-3151299ec91f admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:40.266710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3294ba-f16e-4df6-bc5e-3151299ec91f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:40.268383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3294ba-f16e-4df6-bc5e-3151299ec91f admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:40.279010 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1167/2336] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:02:40.283672 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16caa5f-8b38-40a2-b5b8-612460406569 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:40.289465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:40.299707 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16caa5f-8b38-40a2-b5b8-612460406569 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:40.307682 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029742f8-bf8d-437d-900c-d7538b44556c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:40.308891 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029742f8-bf8d-437d-900c-d7538b44556c admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:40.309134 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029742f8-bf8d-437d-900c-d7538b44556c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:40.309421 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029742f8-bf8d-437d-900c-d7538b44556c admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:40.310440 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029742f8-bf8d-437d-900c-d7538b44556c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:40.312569 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029742f8-bf8d-437d-900c-d7538b44556c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:40.324281 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1170/2337] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:40.328855 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3294ba-f16e-4df6-bc5e-3151299ec91f admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:40.339507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3294ba-f16e-4df6-bc5e-3151299ec91f admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:40.345615 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e83b25b-4484-4297-b48a-36483aba99c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:40.346733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e83b25b-4484-4297-b48a-36483aba99c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:40.347074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e83b25b-4484-4297-b48a-36483aba99c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:40.347412 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e83b25b-4484-4297-b48a-36483aba99c2 admin admin] PATH_INFO: `/v3/projects/3d6a59e3e40a4c888b6f8a71f4142e5c/users/a59f9aec00c0401eb59535c21a10eb00/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:40.363929 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e83b25b-4484-4297-b48a-36483aba99c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d6a59e3e40a4c888b6f8a71f4142e5c, user_id=a59f9aec00c0401eb59535c21a10eb00, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:40.366876 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e83b25b-4484-4297-b48a-36483aba99c2 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:40.385493 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1168/2338] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:40 2022] PUT /identity/v3/projects/3d6a59e3e40a4c888b6f8a71f4142e5c/users/a59f9aec00c0401eb59535c21a10eb00/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:02:40.396470 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f29e9-e2e0-4cb6-8169-639d43104f66 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:40.396862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f29e9-e2e0-4cb6-8169-639d43104f66 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:40.397249 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f29e9-e2e0-4cb6-8169-639d43104f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:40.428266 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-888f29e9-e2e0-4cb6-8169-639d43104f66 None None] MFA Rules not processed for user `a59f9aec00c0401eb59535c21a10eb00`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:40.429541 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-888f29e9-e2e0-4cb6-8169-639d43104f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:40.444651 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e83b25b-4484-4297-b48a-36483aba99c2 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:40.446397 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-3e83b25b-4484-4297-b48a-36483aba99c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:40.505601 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1171/2339] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:40.512579 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f29e9-e2e0-4cb6-8169-639d43104f66 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:40.537277 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e83b25b-4484-4297-b48a-36483aba99c2 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:40.560022 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3364f-92e5-4a28-ad64-995513f85cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:40.561096 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3364f-92e5-4a28-ad64-995513f85cf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:40.561577 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3364f-92e5-4a28-ad64-995513f85cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:40.562342 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3364f-92e5-4a28-ad64-995513f85cf7 admin admin] PATH_INFO: `/v3/projects/33739a601f854277ae1931e8c4cd14aa` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:40.563888 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3364f-92e5-4a28-ad64-995513f85cf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33739a601f854277ae1931e8c4cd14aa)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:40.569500 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3364f-92e5-4a28-ad64-995513f85cf7 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:40.595214 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f29e9-e2e0-4cb6-8169-639d43104f66 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:40.601438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:40.603046 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f5081-62ac-4007-849c-d90daeeb1e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:40.603835 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f5081-62ac-4007-849c-d90daeeb1e39 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:40.604376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f5081-62ac-4007-849c-d90daeeb1e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:40.604808 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f5081-62ac-4007-849c-d90daeeb1e39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:40.605605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f5081-62ac-4007-849c-d90daeeb1e39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:40.607806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f5081-62ac-4007-849c-d90daeeb1e39 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:40.634503 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-64b3364f-92e5-4a28-ad64-995513f85cf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33739a601f854277ae1931e8c4cd14aa', 'request_id': 'req-64b3364f-92e5-4a28-ad64-995513f85cf7'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:40.645328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1172/2340] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:02:40 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:40.646822 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f5081-62ac-4007-849c-d90daeeb1e39 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:40.652329 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-64b3364f-92e5-4a28-ad64-995513f85cf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33739a601f854277ae1931e8c4cd14aa', 'request_id': 'req-64b3364f-92e5-4a28-ad64-995513f85cf7'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:40.660741 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-64b3364f-92e5-4a28-ad64-995513f85cf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33739a601f854277ae1931e8c4cd14aa', 'request_id': 'req-64b3364f-92e5-4a28-ad64-995513f85cf7'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:40.671416 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-64b3364f-92e5-4a28-ad64-995513f85cf7 admin admin] The token cache is being invalidate because project 33739a601f854277ae1931e8c4cd14aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:40.671865 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-64b3364f-92e5-4a28-ad64-995513f85cf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:40.672604 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-64b3364f-92e5-4a28-ad64-995513f85cf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:40.687600 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1169/2341] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:40 2022] DELETE /identity/v3/projects/33739a601f854277ae1931e8c4cd14aa => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:02:40.748492 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f5081-62ac-4007-849c-d90daeeb1e39 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:40.755851 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb86ec-de94-401b-93b8-d881ba40dcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:40.756583 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb86ec-de94-401b-93b8-d881ba40dcac admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:40.756875 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb86ec-de94-401b-93b8-d881ba40dcac admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:40.757155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb86ec-de94-401b-93b8-d881ba40dcac admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:40.757859 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb86ec-de94-401b-93b8-d881ba40dcac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:40.761073 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb86ec-de94-401b-93b8-d881ba40dcac admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:40.776173 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:40.795718 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1173/2342] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:02:40 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:02:40.798464 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3364f-92e5-4a28-ad64-995513f85cf7 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:40.802092 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-64b3364f-92e5-4a28-ad64-995513f85cf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:40.881519 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3364f-92e5-4a28-ad64-995513f85cf7 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:40.886780 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ee72a-a6e7-402e-9592-fe75b33ad9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:40.887424 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ee72a-a6e7-402e-9592-fe75b33ad9d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:40.887773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ee72a-a6e7-402e-9592-fe75b33ad9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:40.888048 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ee72a-a6e7-402e-9592-fe75b33ad9d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:40.888803 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6ee72a-a6e7-402e-9592-fe75b33ad9d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:40.890819 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6ee72a-a6e7-402e-9592-fe75b33ad9d1 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:40.906424 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1170/2343] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:40.910604 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb86ec-de94-401b-93b8-d881ba40dcac admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:40.921114 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb86ec-de94-401b-93b8-d881ba40dcac admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:40.925819 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb6636-2075-4b23-9ecc-4b012628e61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:40.926448 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb6636-2075-4b23-9ecc-4b012628e61c admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:40.926725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb6636-2075-4b23-9ecc-4b012628e61c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:40.926997 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb6636-2075-4b23-9ecc-4b012628e61c admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:40.927752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eb6636-2075-4b23-9ecc-4b012628e61c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:40.929351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eb6636-2075-4b23-9ecc-4b012628e61c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:40.943624 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1174/2344] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:40.945120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ee72a-a6e7-402e-9592-fe75b33ad9d1 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:40.956085 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ee72a-a6e7-402e-9592-fe75b33ad9d1 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:40.965603 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414a55d-d947-4aff-98e2-17694a2701bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:40.966291 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414a55d-d947-4aff-98e2-17694a2701bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:40.966574 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414a55d-d947-4aff-98e2-17694a2701bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:40.966853 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414a55d-d947-4aff-98e2-17694a2701bb admin admin] PATH_INFO: `/v3/projects/1ba7557a99d34df0a9d5f538d930bb0d/users/49a92907c6324b05b155e3c128dd475c/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:40.982272 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414a55d-d947-4aff-98e2-17694a2701bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ba7557a99d34df0a9d5f538d930bb0d, user_id=49a92907c6324b05b155e3c128dd475c, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:40.985070 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414a55d-d947-4aff-98e2-17694a2701bb admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:40.999619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1171/2345] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:40 2022] PUT /identity/v3/projects/1ba7557a99d34df0a9d5f538d930bb0d/users/49a92907c6324b05b155e3c128dd475c/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:41.010589 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f7e46-ce28-4545-a343-9655d8bef40e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:41.011150 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f7e46-ce28-4545-a343-9655d8bef40e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:41.011266 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f7e46-ce28-4545-a343-9655d8bef40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:41.040141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6a7f7e46-ce28-4545-a343-9655d8bef40e None None] MFA Rules not processed for user `49a92907c6324b05b155e3c128dd475c`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:41.041367 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6a7f7e46-ce28-4545-a343-9655d8bef40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:41.124407 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1175/2346] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:41.129610 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414a55d-d947-4aff-98e2-17694a2701bb admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:41.200801 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414a55d-d947-4aff-98e2-17694a2701bb admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:41.207531 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d4746-1148-4ae6-a1e1-f9423e4fed99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:41.208183 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d4746-1148-4ae6-a1e1-f9423e4fed99 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:41.208463 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d4746-1148-4ae6-a1e1-f9423e4fed99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:41.208736 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d4746-1148-4ae6-a1e1-f9423e4fed99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:41.209403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031d4746-1148-4ae6-a1e1-f9423e4fed99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:41.211326 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031d4746-1148-4ae6-a1e1-f9423e4fed99 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:41.240563 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1172/2347] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:02:41 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:41.245601 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f7e46-ce28-4545-a343-9655d8bef40e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:41.256763 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f7e46-ce28-4545-a343-9655d8bef40e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:41.313058 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414da2b0-0e09-4724-bc92-ad00b197d0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:41.313688 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414da2b0-0e09-4724-bc92-ad00b197d0ec admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:41.313971 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414da2b0-0e09-4724-bc92-ad00b197d0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:41.314238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414da2b0-0e09-4724-bc92-ad00b197d0ec admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:41.314908 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414da2b0-0e09-4724-bc92-ad00b197d0ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:41.316862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414da2b0-0e09-4724-bc92-ad00b197d0ec admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:41.340461 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1176/2348] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:02:41 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:02:41.344709 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d4746-1148-4ae6-a1e1-f9423e4fed99 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:41.355348 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d4746-1148-4ae6-a1e1-f9423e4fed99 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:41.360302 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e960c2d-9cfb-491e-8f73-dfbee7f9d687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:41.360920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e960c2d-9cfb-491e-8f73-dfbee7f9d687 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:41.361200 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e960c2d-9cfb-491e-8f73-dfbee7f9d687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:41.361472 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e960c2d-9cfb-491e-8f73-dfbee7f9d687 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:41.362165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e960c2d-9cfb-491e-8f73-dfbee7f9d687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:41.363815 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e960c2d-9cfb-491e-8f73-dfbee7f9d687 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:41.366075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:41.377009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1173/2349] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:41.382895 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414da2b0-0e09-4724-bc92-ad00b197d0ec admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:41.393447 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414da2b0-0e09-4724-bc92-ad00b197d0ec admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:41.399642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93532a6d-e927-420f-80f6-b67338223fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:41.400336 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93532a6d-e927-420f-80f6-b67338223fc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:41.400619 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93532a6d-e927-420f-80f6-b67338223fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:41.400892 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93532a6d-e927-420f-80f6-b67338223fc4 admin admin] PATH_INFO: `/v3/projects/86aa071b2233404abbbd53decf22a8f0/users/df92fe4628614d8ba15d1b67eff3e5b5/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:41.417247 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93532a6d-e927-420f-80f6-b67338223fc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86aa071b2233404abbbd53decf22a8f0, user_id=df92fe4628614d8ba15d1b67eff3e5b5, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:41.419968 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93532a6d-e927-420f-80f6-b67338223fc4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:41.439033 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1177/2350] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:41 2022] PUT /identity/v3/projects/86aa071b2233404abbbd53decf22a8f0/users/df92fe4628614d8ba15d1b67eff3e5b5/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:41.443317 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e960c2d-9cfb-491e-8f73-dfbee7f9d687 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:41.454544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e960c2d-9cfb-491e-8f73-dfbee7f9d687 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:41.518300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be06b8b-94b3-47b3-90ca-615b3f43bc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:41.518839 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be06b8b-94b3-47b3-90ca-615b3f43bc8e admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:41.519122 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be06b8b-94b3-47b3-90ca-615b3f43bc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:41.519403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be06b8b-94b3-47b3-90ca-615b3f43bc8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:41.520152 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be06b8b-94b3-47b3-90ca-615b3f43bc8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:41.521769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be06b8b-94b3-47b3-90ca-615b3f43bc8e admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:41.532476 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1174/2351] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:02:41.536880 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93532a6d-e927-420f-80f6-b67338223fc4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:41.548459 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93532a6d-e927-420f-80f6-b67338223fc4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:41.553814 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf2088-23a7-477d-af11-554dbd31a14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:41.554522 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf2088-23a7-477d-af11-554dbd31a14d admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:41.554805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf2088-23a7-477d-af11-554dbd31a14d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:41.555072 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf2088-23a7-477d-af11-554dbd31a14d admin admin] PATH_INFO: `/v3/projects/86aa071b2233404abbbd53decf22a8f0/users/df92fe4628614d8ba15d1b67eff3e5b5/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:41.557191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf2088-23a7-477d-af11-554dbd31a14d admin admin] RBAC: Authorizing `identity:create_grant(project_id=86aa071b2233404abbbd53decf22a8f0, user_id=df92fe4628614d8ba15d1b67eff3e5b5, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:41.560030 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf2088-23a7-477d-af11-554dbd31a14d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:41.572576 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1178/2352] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:41 2022] PUT /identity/v3/projects/86aa071b2233404abbbd53decf22a8f0/users/df92fe4628614d8ba15d1b67eff3e5b5/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:41.583036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d2e19-e35a-4231-9e76-169578a47aae None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:41.583328 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d2e19-e35a-4231-9e76-169578a47aae None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:41.583685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d2e19-e35a-4231-9e76-169578a47aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:41.617504 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-110d2e19-e35a-4231-9e76-169578a47aae None None] MFA Rules not processed for user `df92fe4628614d8ba15d1b67eff3e5b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:41.618768 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-110d2e19-e35a-4231-9e76-169578a47aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:41.694981 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1175/2353] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:41.706915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf2088-23a7-477d-af11-554dbd31a14d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:41.708733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f9bf2088-23a7-477d-af11-554dbd31a14d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:41.740777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:41.771193 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf2088-23a7-477d-af11-554dbd31a14d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:41.775335 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c81d64-067a-449c-bd38-bead966b4207 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:41.776009 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c81d64-067a-449c-bd38-bead966b4207 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:41.776303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c81d64-067a-449c-bd38-bead966b4207 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:41.776572 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c81d64-067a-449c-bd38-bead966b4207 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:41.778758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-43c81d64-067a-449c-bd38-bead966b4207 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:41.841388 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c81d64-067a-449c-bd38-bead966b4207 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:41.842824 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c81d64-067a-449c-bd38-bead966b4207 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:41.854793 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1179/2354] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:02:44.494134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d2e19-e35a-4231-9e76-169578a47aae None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:44.504716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d2e19-e35a-4231-9e76-169578a47aae None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:44.508649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f7ee6a-6b2b-448d-a67a-5ae9761d6ef9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:44.509239 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7ee6a-6b2b-448d-a67a-5ae9761d6ef9 service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:44.509520 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7ee6a-6b2b-448d-a67a-5ae9761d6ef9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:44.509790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7ee6a-6b2b-448d-a67a-5ae9761d6ef9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:44.569454 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f7ee6a-6b2b-448d-a67a-5ae9761d6ef9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:44.570889 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f7ee6a-6b2b-448d-a67a-5ae9761d6ef9 service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:44.583173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1176/2355] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:02:45.303663 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c81d64-067a-449c-bd38-bead966b4207 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:45.305828 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-43c81d64-067a-449c-bd38-bead966b4207 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:45.317374 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c81d64-067a-449c-bd38-bead966b4207 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:45.321295 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02385ee8-bf70-43d3-8d37-22b7866d5884 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:45.322305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02385ee8-bf70-43d3-8d37-22b7866d5884 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:45.322687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02385ee8-bf70-43d3-8d37-22b7866d5884 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:45.322892 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02385ee8-bf70-43d3-8d37-22b7866d5884 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:45.333800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02385ee8-bf70-43d3-8d37-22b7866d5884 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:45.335348 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02385ee8-bf70-43d3-8d37-22b7866d5884 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:45.351486 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1180/2356] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:45 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:02:46.578254 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f7ee6a-6b2b-448d-a67a-5ae9761d6ef9 service cinder] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:46.579962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-63f7ee6a-6b2b-448d-a67a-5ae9761d6ef9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:46.607542 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:46.672937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f7ee6a-6b2b-448d-a67a-5ae9761d6ef9 service cinder] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:46.679604 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fcee76-55f1-409d-abbb-91adfcc8b846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:46.680281 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fcee76-55f1-409d-abbb-91adfcc8b846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:46.680575 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fcee76-55f1-409d-abbb-91adfcc8b846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:46.681180 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fcee76-55f1-409d-abbb-91adfcc8b846 admin admin] PATH_INFO: `/v3/users/86cba820b8c148848e165bedcafb577a` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:46.683021 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fcee76-55f1-409d-abbb-91adfcc8b846 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86cba820b8c148848e165bedcafb577a)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:46.685190 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fcee76-55f1-409d-abbb-91adfcc8b846 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:46.758207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-71fcee76-55f1-409d-abbb-91adfcc8b846 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86cba820b8c148848e165bedcafb577a'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:46.766201 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-71fcee76-55f1-409d-abbb-91adfcc8b846 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86cba820b8c148848e165bedcafb577a'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:46.774237 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-71fcee76-55f1-409d-abbb-91adfcc8b846 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86cba820b8c148848e165bedcafb577a'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:46.774895 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-71fcee76-55f1-409d-abbb-91adfcc8b846 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86cba820b8c148848e165bedcafb577a'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:46.775702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-71fcee76-55f1-409d-abbb-91adfcc8b846 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86cba820b8c148848e165bedcafb577a'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:46.795625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1177/2357] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:02:46 2022] DELETE /identity/v3/users/86cba820b8c148848e165bedcafb577a => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:46.800821 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c38fc-4d79-4faa-95d2-1bf774774360 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:46.801390 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c38fc-4d79-4faa-95d2-1bf774774360 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:46.801647 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c38fc-4d79-4faa-95d2-1bf774774360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:46.872812 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9a6c38fc-4d79-4faa-95d2-1bf774774360 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:46.874091 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9a6c38fc-4d79-4faa-95d2-1bf774774360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:46.995498 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1181/2358] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:47.007113 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fcee76-55f1-409d-abbb-91adfcc8b846 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:47.009606 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-71fcee76-55f1-409d-abbb-91adfcc8b846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:47.098270 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fcee76-55f1-409d-abbb-91adfcc8b846 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:47.102941 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094e181-7e3f-4ca9-8cf0-93c8b90b59e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:47.103669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094e181-7e3f-4ca9-8cf0-93c8b90b59e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:47.103949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094e181-7e3f-4ca9-8cf0-93c8b90b59e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:47.104216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094e181-7e3f-4ca9-8cf0-93c8b90b59e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:47.117466 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094e181-7e3f-4ca9-8cf0-93c8b90b59e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:47.117945 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094e181-7e3f-4ca9-8cf0-93c8b90b59e3 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:47.130093 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1178/2359] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:02:47.505012 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c38fc-4d79-4faa-95d2-1bf774774360 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:47.507198 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9a6c38fc-4d79-4faa-95d2-1bf774774360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:47.521160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c38fc-4d79-4faa-95d2-1bf774774360 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:47.526946 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61412b5-b33b-4fa0-8b41-f9c6bc601e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:47.527859 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61412b5-b33b-4fa0-8b41-f9c6bc601e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:47.528209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61412b5-b33b-4fa0-8b41-f9c6bc601e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:47.528495 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61412b5-b33b-4fa0-8b41-f9c6bc601e46 admin admin] PATH_INFO: `/v3/projects/f240e3cde14144aeb8e8582c66ed8d8c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:47.530006 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61412b5-b33b-4fa0-8b41-f9c6bc601e46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f240e3cde14144aeb8e8582c66ed8d8c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:47.532116 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61412b5-b33b-4fa0-8b41-f9c6bc601e46 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:47.591054 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a61412b5-b33b-4fa0-8b41-f9c6bc601e46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f240e3cde14144aeb8e8582c66ed8d8c', 'request_id': 'req-a61412b5-b33b-4fa0-8b41-f9c6bc601e46'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:47.598043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a61412b5-b33b-4fa0-8b41-f9c6bc601e46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f240e3cde14144aeb8e8582c66ed8d8c', 'request_id': 'req-a61412b5-b33b-4fa0-8b41-f9c6bc601e46'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:47.608768 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a61412b5-b33b-4fa0-8b41-f9c6bc601e46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f240e3cde14144aeb8e8582c66ed8d8c', 'request_id': 'req-a61412b5-b33b-4fa0-8b41-f9c6bc601e46'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:47.611159 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:47.621958 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a61412b5-b33b-4fa0-8b41-f9c6bc601e46 admin admin] The token cache is being invalidate because project f240e3cde14144aeb8e8582c66ed8d8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:47.622256 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a61412b5-b33b-4fa0-8b41-f9c6bc601e46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:47.622945 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a61412b5-b33b-4fa0-8b41-f9c6bc601e46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:47.624597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1182/2360] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:47 2022] DELETE /identity/v3/projects/f240e3cde14144aeb8e8582c66ed8d8c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:47.629767 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e7989-ac3a-46e9-a069-ca52299f911f None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:47.630273 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e7989-ac3a-46e9-a069-ca52299f911f None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:47.630380 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e7989-ac3a-46e9-a069-ca52299f911f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:47.651261 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-9c0e7989-ac3a-46e9-a069-ca52299f911f None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:47.652490 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-9c0e7989-ac3a-46e9-a069-ca52299f911f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:47.688921 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:47.717746 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1179/2361] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:47.725881 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bef00a-78b2-420d-a991-d595a93c801a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:47.726169 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bef00a-78b2-420d-a991-d595a93c801a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:47.726445 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bef00a-78b2-420d-a991-d595a93c801a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:47.747229 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a9bef00a-78b2-420d-a991-d595a93c801a None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:47.748493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a9bef00a-78b2-420d-a991-d595a93c801a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:47.758195 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1183/2362] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:02:47.762778 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e7989-ac3a-46e9-a069-ca52299f911f None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:47.773034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e7989-ac3a-46e9-a069-ca52299f911f None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:47.778342 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1f1e3-2e1c-4280-ab3d-fba57a0c557d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:47.778992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1f1e3-2e1c-4280-ab3d-fba57a0c557d admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:47.779271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1f1e3-2e1c-4280-ab3d-fba57a0c557d admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:47.779563 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1f1e3-2e1c-4280-ab3d-fba57a0c557d admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:47.780283 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1f1e3-2e1c-4280-ab3d-fba57a0c557d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:02:47.780613 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1f1e3-2e1c-4280-ab3d-fba57a0c557d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:47.782346 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1f1e3-2e1c-4280-ab3d-fba57a0c557d admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:47.799634 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1180/2363] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:02:47.801057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bef00a-78b2-420d-a991-d595a93c801a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:47.810756 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bef00a-78b2-420d-a991-d595a93c801a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:47.816102 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d7b4b-5487-460f-ac39-e7c924651ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:47.816736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d7b4b-5487-460f-ac39-e7c924651ae8 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:47.817012 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d7b4b-5487-460f-ac39-e7c924651ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:47.817286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d7b4b-5487-460f-ac39-e7c924651ae8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:47.817947 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d7b4b-5487-460f-ac39-e7c924651ae8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:47.819932 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d7b4b-5487-460f-ac39-e7c924651ae8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:47.853675 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1184/2364] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:02:47 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:47.854768 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1f1e3-2e1c-4280-ab3d-fba57a0c557d admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:47.865631 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1f1e3-2e1c-4280-ab3d-fba57a0c557d admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:47.929220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7902d33-d850-481d-8155-893f1010f2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:47.929984 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7902d33-d850-481d-8155-893f1010f2f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:47.930329 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7902d33-d850-481d-8155-893f1010f2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:47.930605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7902d33-d850-481d-8155-893f1010f2f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:47.931284 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7902d33-d850-481d-8155-893f1010f2f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:47.933256 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7902d33-d850-481d-8155-893f1010f2f9 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:47.961454 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1181/2365] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:02:47 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:02:47.962643 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d7b4b-5487-460f-ac39-e7c924651ae8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:47.973625 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d7b4b-5487-460f-ac39-e7c924651ae8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:47.978478 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6122d-a360-4210-baf7-c0e614544cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:47.979106 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6122d-a360-4210-baf7-c0e614544cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:47.979368 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6122d-a360-4210-baf7-c0e614544cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:47.979667 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6122d-a360-4210-baf7-c0e614544cc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:47.980541 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f6122d-a360-4210-baf7-c0e614544cc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:47.981973 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f6122d-a360-4210-baf7-c0e614544cc2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:47.994775 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1185/2366] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:02:47.999037 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7902d33-d850-481d-8155-893f1010f2f9 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:48.012807 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7902d33-d850-481d-8155-893f1010f2f9 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:48.017369 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974bd3a-a96d-47c4-a653-90c2b86bcbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:48.017993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974bd3a-a96d-47c4-a653-90c2b86bcbff admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:48.018277 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974bd3a-a96d-47c4-a653-90c2b86bcbff admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:48.018551 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974bd3a-a96d-47c4-a653-90c2b86bcbff admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:48.019252 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d974bd3a-a96d-47c4-a653-90c2b86bcbff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:48.020935 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d974bd3a-a96d-47c4-a653-90c2b86bcbff admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:48.035743 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1182/2367] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:48.037889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6122d-a360-4210-baf7-c0e614544cc2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:48.050801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6122d-a360-4210-baf7-c0e614544cc2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:48.056562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b3d72-4526-4764-8e7b-8a49575e397b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:48.057248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b3d72-4526-4764-8e7b-8a49575e397b admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:48.057530 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b3d72-4526-4764-8e7b-8a49575e397b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:48.057800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b3d72-4526-4764-8e7b-8a49575e397b admin admin] PATH_INFO: `/v3/projects/3ccf078eafe84a84ac66518c5ff4fc1e/users/61a4b34360f24208980ffb297e87d2e9/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:48.076138 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b3d72-4526-4764-8e7b-8a49575e397b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ccf078eafe84a84ac66518c5ff4fc1e, user_id=61a4b34360f24208980ffb297e87d2e9, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:48.080572 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b3d72-4526-4764-8e7b-8a49575e397b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:48.095599 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1186/2368] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:48 2022] PUT /identity/v3/projects/3ccf078eafe84a84ac66518c5ff4fc1e/users/61a4b34360f24208980ffb297e87d2e9/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:48.173242 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca0b8b-ab13-4d95-8f2a-dce849dce5be None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:48.173539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca0b8b-ab13-4d95-8f2a-dce849dce5be None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:48.175088 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca0b8b-ab13-4d95-8f2a-dce849dce5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:48.243217 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-0fca0b8b-ab13-4d95-8f2a-dce849dce5be None None] MFA Rules not processed for user `61a4b34360f24208980ffb297e87d2e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:48.244802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-0fca0b8b-ab13-4d95-8f2a-dce849dce5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:48.333377 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1183/2369] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:48 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:48.343767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b3d72-4526-4764-8e7b-8a49575e397b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:48.345622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4d9b3d72-4526-4764-8e7b-8a49575e397b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:48.417294 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b3d72-4526-4764-8e7b-8a49575e397b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:48.422707 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a7839-7f37-4287-9236-e811f31a5102 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:48.423341 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a7839-7f37-4287-9236-e811f31a5102 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:48.423811 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a7839-7f37-4287-9236-e811f31a5102 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:48.424260 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a7839-7f37-4287-9236-e811f31a5102 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:48.439959 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a7839-7f37-4287-9236-e811f31a5102 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:48.439959 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a7839-7f37-4287-9236-e811f31a5102 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:48.445941 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1187/2370] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:48 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:02:48.896021 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e33fd-8987-4d5d-b942-6554a59a9017 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:48.896583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e33fd-8987-4d5d-b942-6554a59a9017 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:48.896583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e33fd-8987-4d5d-b942-6554a59a9017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:48.927335 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-fa8e33fd-8987-4d5d-b942-6554a59a9017 None None] MFA Rules not processed for user `3a8f3eb811ae40ad9cb64261f95a992e`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:48.928640 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-fa8e33fd-8987-4d5d-b942-6554a59a9017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:48.988847 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:49.017252 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1184/2371] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jul 26 05:02:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:49.067408 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a7839-7f37-4287-9236-e811f31a5102 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:49.068860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b37a7839-7f37-4287-9236-e811f31a5102 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:49.141239 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a7839-7f37-4287-9236-e811f31a5102 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:49.146702 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0222e9-3672-465f-851b-67b3e4e7d510 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5218319aaa5424882714bffcbeee730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:49.147345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0222e9-3672-465f-851b-67b3e4e7d510 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:49.147665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0222e9-3672-465f-851b-67b3e4e7d510 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:49.147947 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0222e9-3672-465f-851b-67b3e4e7d510 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:49.157015 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0222e9-3672-465f-851b-67b3e4e7d510 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:49.158608 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0222e9-3672-465f-851b-67b3e4e7d510 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:49.168033 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1188/2372] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jul 26 05:02:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 05:02:49.540168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e33fd-8987-4d5d-b942-6554a59a9017 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:49.541900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-fa8e33fd-8987-4d5d-b942-6554a59a9017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:49.655885 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e33fd-8987-4d5d-b942-6554a59a9017 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:49.669483 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991cae6-c41a-421d-a339-1a3170381693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:49.670544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991cae6-c41a-421d-a339-1a3170381693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:49.671012 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991cae6-c41a-421d-a339-1a3170381693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:49.671437 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991cae6-c41a-421d-a339-1a3170381693 admin admin] PATH_INFO: `/v3/users/a59f9aec00c0401eb59535c21a10eb00` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:49.674063 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991cae6-c41a-421d-a339-1a3170381693 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a59f9aec00c0401eb59535c21a10eb00)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:49.678133 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991cae6-c41a-421d-a339-1a3170381693 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:49.776375 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-4991cae6-c41a-421d-a339-1a3170381693 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a59f9aec00c0401eb59535c21a10eb00'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:49.796647 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-4991cae6-c41a-421d-a339-1a3170381693 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a59f9aec00c0401eb59535c21a10eb00'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:49.835703 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-4991cae6-c41a-421d-a339-1a3170381693 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a59f9aec00c0401eb59535c21a10eb00'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:49.835703 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-4991cae6-c41a-421d-a339-1a3170381693 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a59f9aec00c0401eb59535c21a10eb00'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:49.835703 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-4991cae6-c41a-421d-a339-1a3170381693 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a59f9aec00c0401eb59535c21a10eb00'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:49.875062 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1185/2373] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:02:49 2022] DELETE /identity/v3/users/a59f9aec00c0401eb59535c21a10eb00 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:49.879521 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad426eb4-6b29-41a0-8e5b-2bd108889b1b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:49.880055 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad426eb4-6b29-41a0-8e5b-2bd108889b1b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:49.880434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad426eb4-6b29-41a0-8e5b-2bd108889b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:49.937394 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ad426eb4-6b29-41a0-8e5b-2bd108889b1b None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:49.939376 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ad426eb4-6b29-41a0-8e5b-2bd108889b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:49.974255 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:50.066329 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1189/2374] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:50.077666 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991cae6-c41a-421d-a339-1a3170381693 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:50.079863 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-4991cae6-c41a-421d-a339-1a3170381693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:50.195042 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991cae6-c41a-421d-a339-1a3170381693 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:50.204857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0b840-d89f-4ee9-8f5c-d85961b053ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:50.205614 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0b840-d89f-4ee9-8f5c-d85961b053ba service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:50.205838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0b840-d89f-4ee9-8f5c-d85961b053ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:50.206077 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0b840-d89f-4ee9-8f5c-d85961b053ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:50.228282 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0b840-d89f-4ee9-8f5c-d85961b053ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:50.229861 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0b840-d89f-4ee9-8f5c-d85961b053ba service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:50.258920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1186/2375] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:02:50.855848 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad426eb4-6b29-41a0-8e5b-2bd108889b1b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:50.857890 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ad426eb4-6b29-41a0-8e5b-2bd108889b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:50.885519 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad426eb4-6b29-41a0-8e5b-2bd108889b1b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:50.901236 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e695d7e-5505-4673-853c-59e9b05f7654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:50.902233 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e695d7e-5505-4673-853c-59e9b05f7654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:50.902624 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e695d7e-5505-4673-853c-59e9b05f7654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:50.902982 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e695d7e-5505-4673-853c-59e9b05f7654 admin admin] PATH_INFO: `/v3/projects/3d6a59e3e40a4c888b6f8a71f4142e5c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:50.904330 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e695d7e-5505-4673-853c-59e9b05f7654 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d6a59e3e40a4c888b6f8a71f4142e5c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:50.906659 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e695d7e-5505-4673-853c-59e9b05f7654 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:50.994265 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5e695d7e-5505-4673-853c-59e9b05f7654 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d6a59e3e40a4c888b6f8a71f4142e5c', 'request_id': 'req-5e695d7e-5505-4673-853c-59e9b05f7654'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:51.007090 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5e695d7e-5505-4673-853c-59e9b05f7654 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d6a59e3e40a4c888b6f8a71f4142e5c', 'request_id': 'req-5e695d7e-5505-4673-853c-59e9b05f7654'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:51.017602 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5e695d7e-5505-4673-853c-59e9b05f7654 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d6a59e3e40a4c888b6f8a71f4142e5c', 'request_id': 'req-5e695d7e-5505-4673-853c-59e9b05f7654'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:51.036922 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5e695d7e-5505-4673-853c-59e9b05f7654 admin admin] The token cache is being invalidate because project 3d6a59e3e40a4c888b6f8a71f4142e5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:51.037212 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5e695d7e-5505-4673-853c-59e9b05f7654 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:51.038616 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5e695d7e-5505-4673-853c-59e9b05f7654 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:51.043441 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1190/2376] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:50 2022] DELETE /identity/v3/projects/3d6a59e3e40a4c888b6f8a71f4142e5c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:51.044903 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0b840-d89f-4ee9-8f5c-d85961b053ba service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:51.051331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:51.067016 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0b840-d89f-4ee9-8f5c-d85961b053ba service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:51.268003 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6de5b1-6b40-4d80-83e3-f11bc06edf5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:51.272452 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6de5b1-6b40-4d80-83e3-f11bc06edf5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:51.272452 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6de5b1-6b40-4d80-83e3-f11bc06edf5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:51.272452 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6de5b1-6b40-4d80-83e3-f11bc06edf5a admin admin] PATH_INFO: `/v3/users/49a92907c6324b05b155e3c128dd475c` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:51.273939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6de5b1-6b40-4d80-83e3-f11bc06edf5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=49a92907c6324b05b155e3c128dd475c)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:51.276038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6de5b1-6b40-4d80-83e3-f11bc06edf5a admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:51.344468 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-fe6de5b1-6b40-4d80-83e3-f11bc06edf5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49a92907c6324b05b155e3c128dd475c'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:51.354333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-fe6de5b1-6b40-4d80-83e3-f11bc06edf5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49a92907c6324b05b155e3c128dd475c'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:51.363886 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-fe6de5b1-6b40-4d80-83e3-f11bc06edf5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49a92907c6324b05b155e3c128dd475c'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:51.364559 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-fe6de5b1-6b40-4d80-83e3-f11bc06edf5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49a92907c6324b05b155e3c128dd475c'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:51.365336 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-fe6de5b1-6b40-4d80-83e3-f11bc06edf5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49a92907c6324b05b155e3c128dd475c'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:51.395051 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1187/2377] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:02:51 2022] DELETE /identity/v3/users/49a92907c6324b05b155e3c128dd475c => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:52.032257 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e695d7e-5505-4673-853c-59e9b05f7654 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:52.034211 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5e695d7e-5505-4673-853c-59e9b05f7654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:52.056512 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e695d7e-5505-4673-853c-59e9b05f7654 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:52.144804 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd81ab-4632-479a-a590-783644cd2c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:52.145481 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd81ab-4632-479a-a590-783644cd2c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:52.145766 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd81ab-4632-479a-a590-783644cd2c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:52.146163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd81ab-4632-479a-a590-783644cd2c55 admin admin] PATH_INFO: `/v3/projects/1ba7557a99d34df0a9d5f538d930bb0d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:52.147234 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cd81ab-4632-479a-a590-783644cd2c55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ba7557a99d34df0a9d5f538d930bb0d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:52.149575 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cd81ab-4632-479a-a590-783644cd2c55 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:52.220926 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:52.228909 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-98cd81ab-4632-479a-a590-783644cd2c55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ba7557a99d34df0a9d5f538d930bb0d', 'request_id': 'req-98cd81ab-4632-479a-a590-783644cd2c55'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:52.238091 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-98cd81ab-4632-479a-a590-783644cd2c55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ba7557a99d34df0a9d5f538d930bb0d', 'request_id': 'req-98cd81ab-4632-479a-a590-783644cd2c55'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:52.251362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-98cd81ab-4632-479a-a590-783644cd2c55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ba7557a99d34df0a9d5f538d930bb0d', 'request_id': 'req-98cd81ab-4632-479a-a590-783644cd2c55'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:52.268125 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-98cd81ab-4632-479a-a590-783644cd2c55 admin admin] The token cache is being invalidate because project 1ba7557a99d34df0a9d5f538d930bb0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:52.269420 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-98cd81ab-4632-479a-a590-783644cd2c55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:52.270262 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-98cd81ab-4632-479a-a590-783644cd2c55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:52.276069 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1191/2378] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:52 2022] DELETE /identity/v3/projects/1ba7557a99d34df0a9d5f538d930bb0d => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:52.276533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6de5b1-6b40-4d80-83e3-f11bc06edf5a admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:52.278897 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-fe6de5b1-6b40-4d80-83e3-f11bc06edf5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:52.297630 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6de5b1-6b40-4d80-83e3-f11bc06edf5a admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:52.385389 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21947513-60c1-4350-9860-cbcc55c2499a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:52.386360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21947513-60c1-4350-9860-cbcc55c2499a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:52.386712 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21947513-60c1-4350-9860-cbcc55c2499a admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:52.386996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21947513-60c1-4350-9860-cbcc55c2499a admin admin] PATH_INFO: `/v3/users/df92fe4628614d8ba15d1b67eff3e5b5` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:52.388640 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21947513-60c1-4350-9860-cbcc55c2499a admin admin] RBAC: Authorizing `identity:delete_user(user_id=df92fe4628614d8ba15d1b67eff3e5b5)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:52.393830 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21947513-60c1-4350-9860-cbcc55c2499a admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:52.416121 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:52.483898 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-21947513-60c1-4350-9860-cbcc55c2499a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df92fe4628614d8ba15d1b67eff3e5b5'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:52.506911 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-21947513-60c1-4350-9860-cbcc55c2499a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df92fe4628614d8ba15d1b67eff3e5b5'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:52.522834 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-21947513-60c1-4350-9860-cbcc55c2499a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df92fe4628614d8ba15d1b67eff3e5b5'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:52.523610 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-21947513-60c1-4350-9860-cbcc55c2499a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df92fe4628614d8ba15d1b67eff3e5b5'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:52.525457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-21947513-60c1-4350-9860-cbcc55c2499a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df92fe4628614d8ba15d1b67eff3e5b5'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:52.556202 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1188/2379] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:02:52 2022] DELETE /identity/v3/users/df92fe4628614d8ba15d1b67eff3e5b5 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:02:52.947400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd81ab-4632-479a-a590-783644cd2c55 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:52.949767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-98cd81ab-4632-479a-a590-783644cd2c55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:52.971494 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd81ab-4632-479a-a590-783644cd2c55 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:53.076469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2039bb-4392-4402-9934-f8c2dce61665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:53.077262 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2039bb-4392-4402-9934-f8c2dce61665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:53.077626 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2039bb-4392-4402-9934-f8c2dce61665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:53.077976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2039bb-4392-4402-9934-f8c2dce61665 admin admin] PATH_INFO: `/v3/projects/86aa071b2233404abbbd53decf22a8f0` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:53.081564 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2039bb-4392-4402-9934-f8c2dce61665 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86aa071b2233404abbbd53decf22a8f0)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:53.081564 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2039bb-4392-4402-9934-f8c2dce61665 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:53.182706 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-de2039bb-4392-4402-9934-f8c2dce61665 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86aa071b2233404abbbd53decf22a8f0', 'request_id': 'req-de2039bb-4392-4402-9934-f8c2dce61665'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:53.197094 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-de2039bb-4392-4402-9934-f8c2dce61665 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86aa071b2233404abbbd53decf22a8f0', 'request_id': 'req-de2039bb-4392-4402-9934-f8c2dce61665'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:53.210320 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-de2039bb-4392-4402-9934-f8c2dce61665 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86aa071b2233404abbbd53decf22a8f0', 'request_id': 'req-de2039bb-4392-4402-9934-f8c2dce61665'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:53.249172 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-de2039bb-4392-4402-9934-f8c2dce61665 admin admin] The token cache is being invalidate because project 86aa071b2233404abbbd53decf22a8f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:53.249534 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-de2039bb-4392-4402-9934-f8c2dce61665 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:53.250170 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-de2039bb-4392-4402-9934-f8c2dce61665 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:53.255127 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1192/2380] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:52 2022] DELETE /identity/v3/projects/86aa071b2233404abbbd53decf22a8f0 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:53.256986 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64959b58-45e3-4a38-ac8a-f0e08c317e20 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:53.257306 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64959b58-45e3-4a38-ac8a-f0e08c317e20 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:53.257581 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64959b58-45e3-4a38-ac8a-f0e08c317e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:53.280214 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-64959b58-45e3-4a38-ac8a-f0e08c317e20 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:53.281482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-64959b58-45e3-4a38-ac8a-f0e08c317e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:53.352518 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1189/2381] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:53.361636 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f22ff-8df9-479f-838b-e8aaf609e51d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:53.364013 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f22ff-8df9-479f-838b-e8aaf609e51d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:53.364013 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f22ff-8df9-479f-838b-e8aaf609e51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:53.384312 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-706f22ff-8df9-479f-838b-e8aaf609e51d None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:53.385560 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-706f22ff-8df9-479f-838b-e8aaf609e51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:53.398185 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1193/2382] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:53.399440 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64959b58-45e3-4a38-ac8a-f0e08c317e20 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:53.413642 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64959b58-45e3-4a38-ac8a-f0e08c317e20 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:53.418871 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7afe0e4-7b20-4b73-ba23-353b894b28bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:53.419659 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afe0e4-7b20-4b73-ba23-353b894b28bb admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:53.419939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afe0e4-7b20-4b73-ba23-353b894b28bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:53.420220 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afe0e4-7b20-4b73-ba23-353b894b28bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:53.420921 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7afe0e4-7b20-4b73-ba23-353b894b28bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:02:53.421284 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7afe0e4-7b20-4b73-ba23-353b894b28bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:53.422859 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7afe0e4-7b20-4b73-ba23-353b894b28bb admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:53.443603 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1190/2383] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:02:53.447656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f22ff-8df9-479f-838b-e8aaf609e51d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:53.457473 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f22ff-8df9-479f-838b-e8aaf609e51d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:53.462373 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa9ea0-3ed3-4cfd-a158-11e72797edab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:53.463169 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa9ea0-3ed3-4cfd-a158-11e72797edab admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:53.463453 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa9ea0-3ed3-4cfd-a158-11e72797edab admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:53.463776 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa9ea0-3ed3-4cfd-a158-11e72797edab admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:53.465055 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa9ea0-3ed3-4cfd-a158-11e72797edab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:53.466673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa9ea0-3ed3-4cfd-a158-11e72797edab admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:53.511044 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1194/2384] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:02:53 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:53.515441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7afe0e4-7b20-4b73-ba23-353b894b28bb admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:53.545722 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7afe0e4-7b20-4b73-ba23-353b894b28bb admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:53.622823 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36464cc3-c3da-41df-bacf-80ecae065304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:53.623770 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36464cc3-c3da-41df-bacf-80ecae065304 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:53.624667 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36464cc3-c3da-41df-bacf-80ecae065304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:53.624667 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36464cc3-c3da-41df-bacf-80ecae065304 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:53.625256 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36464cc3-c3da-41df-bacf-80ecae065304 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:53.627591 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36464cc3-c3da-41df-bacf-80ecae065304 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:53.665981 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1191/2385] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:02:53 2022] POST /identity/v3/users => generated 293 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:53.670708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa9ea0-3ed3-4cfd-a158-11e72797edab admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:53.689398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa9ea0-3ed3-4cfd-a158-11e72797edab admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:53.725528 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df14b7-cf36-4289-95ec-82ac44f1d734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:53.726471 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df14b7-cf36-4289-95ec-82ac44f1d734 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:53.726967 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df14b7-cf36-4289-95ec-82ac44f1d734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:53.727402 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df14b7-cf36-4289-95ec-82ac44f1d734 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:53.728607 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df14b7-cf36-4289-95ec-82ac44f1d734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:53.731593 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df14b7-cf36-4289-95ec-82ac44f1d734 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:53.743819 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1195/2386] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:53.747865 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36464cc3-c3da-41df-bacf-80ecae065304 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:53.760706 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36464cc3-c3da-41df-bacf-80ecae065304 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:53.778771 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa957a8-16e7-46db-b816-b60067fb08fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:53.779618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa957a8-16e7-46db-b816-b60067fb08fd admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:53.779916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa957a8-16e7-46db-b816-b60067fb08fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:53.780188 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa957a8-16e7-46db-b816-b60067fb08fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:53.780924 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa957a8-16e7-46db-b816-b60067fb08fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:53.782599 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa957a8-16e7-46db-b816-b60067fb08fd admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:53.800009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1192/2387] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:02:53.801592 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df14b7-cf36-4289-95ec-82ac44f1d734 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:53.812873 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df14b7-cf36-4289-95ec-82ac44f1d734 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:53.822420 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859eb66-1c72-4b4f-a702-a730415a3efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:53.823883 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859eb66-1c72-4b4f-a702-a730415a3efd admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:53.824593 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859eb66-1c72-4b4f-a702-a730415a3efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:53.825235 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859eb66-1c72-4b4f-a702-a730415a3efd admin admin] PATH_INFO: `/v3/projects/0b1662fb7a414e3b9c9da636ad028e95/users/fb7018c9b76d4f4ab74ed8e45d9e4bb5/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:53.852843 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e859eb66-1c72-4b4f-a702-a730415a3efd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b1662fb7a414e3b9c9da636ad028e95, user_id=fb7018c9b76d4f4ab74ed8e45d9e4bb5, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:53.855721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e859eb66-1c72-4b4f-a702-a730415a3efd admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:53.870436 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:53.878560 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1196/2388] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:53 2022] PUT /identity/v3/projects/0b1662fb7a414e3b9c9da636ad028e95/users/fb7018c9b76d4f4ab74ed8e45d9e4bb5/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:53.953320 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5233fa-5dd7-4e34-881d-d3705f3d352a None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:53.953547 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5233fa-5dd7-4e34-881d-d3705f3d352a None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:53.953833 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5233fa-5dd7-4e34-881d-d3705f3d352a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:53.974449 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:54.003315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-9c5233fa-5dd7-4e34-881d-d3705f3d352a None None] MFA Rules not processed for user `fb7018c9b76d4f4ab74ed8e45d9e4bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:54.007492 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-9c5233fa-5dd7-4e34-881d-d3705f3d352a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:54.090651 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1193/2389] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:54.097854 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859eb66-1c72-4b4f-a702-a730415a3efd admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:54.099724 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e859eb66-1c72-4b4f-a702-a730415a3efd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:54.189140 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859eb66-1c72-4b4f-a702-a730415a3efd admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:54.194518 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261649e-5317-4ffe-9163-3991f4f94f1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:54.195165 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261649e-5317-4ffe-9163-3991f4f94f1d service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:54.195450 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261649e-5317-4ffe-9163-3991f4f94f1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:54.195756 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261649e-5317-4ffe-9163-3991f4f94f1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:54.207163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7261649e-5317-4ffe-9163-3991f4f94f1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:54.208505 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7261649e-5317-4ffe-9163-3991f4f94f1d service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:54.227367 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1197/2390] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:02:57.193034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5233fa-5dd7-4e34-881d-d3705f3d352a None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:57.195049 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-9c5233fa-5dd7-4e34-881d-d3705f3d352a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:57.307082 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261649e-5317-4ffe-9163-3991f4f94f1d service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:57.310358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5233fa-5dd7-4e34-881d-d3705f3d352a None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:57.315466 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ce96f-4b70-4b05-9e9a-d419f33e702e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:57.316384 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ce96f-4b70-4b05-9e9a-d419f33e702e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:57.316823 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ce96f-4b70-4b05-9e9a-d419f33e702e admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:57.317255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ce96f-4b70-4b05-9e9a-d419f33e702e admin admin] PATH_INFO: `/v3/users/61a4b34360f24208980ffb297e87d2e9` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:57.318652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1ce96f-4b70-4b05-9e9a-d419f33e702e admin admin] RBAC: Authorizing `identity:delete_user(user_id=61a4b34360f24208980ffb297e87d2e9)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:57.320917 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1ce96f-4b70-4b05-9e9a-d419f33e702e admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:57.325503 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261649e-5317-4ffe-9163-3991f4f94f1d service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:57.339781 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134845ce-3f25-49c8-88b8-935a299cfc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:57.339781 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134845ce-3f25-49c8-88b8-935a299cfc6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:57.339781 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134845ce-3f25-49c8-88b8-935a299cfc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:57.339781 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134845ce-3f25-49c8-88b8-935a299cfc6f admin admin] PATH_INFO: `/v3/users/fb7018c9b76d4f4ab74ed8e45d9e4bb5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:57.340117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134845ce-3f25-49c8-88b8-935a299cfc6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb7018c9b76d4f4ab74ed8e45d9e4bb5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:57.341737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134845ce-3f25-49c8-88b8-935a299cfc6f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:57.465174 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-134845ce-3f25-49c8-88b8-935a299cfc6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb7018c9b76d4f4ab74ed8e45d9e4bb5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:57.474934 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-5f1ce96f-4b70-4b05-9e9a-d419f33e702e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61a4b34360f24208980ffb297e87d2e9'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:57.483067 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-134845ce-3f25-49c8-88b8-935a299cfc6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb7018c9b76d4f4ab74ed8e45d9e4bb5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:57.483852 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-134845ce-3f25-49c8-88b8-935a299cfc6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb7018c9b76d4f4ab74ed8e45d9e4bb5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:57.485099 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-5f1ce96f-4b70-4b05-9e9a-d419f33e702e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61a4b34360f24208980ffb297e87d2e9'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:57.505632 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-134845ce-3f25-49c8-88b8-935a299cfc6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb7018c9b76d4f4ab74ed8e45d9e4bb5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:57.506342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-134845ce-3f25-49c8-88b8-935a299cfc6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb7018c9b76d4f4ab74ed8e45d9e4bb5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:57.521801 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-5f1ce96f-4b70-4b05-9e9a-d419f33e702e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61a4b34360f24208980ffb297e87d2e9'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:57.523875 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-5f1ce96f-4b70-4b05-9e9a-d419f33e702e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61a4b34360f24208980ffb297e87d2e9'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:57.526020 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-5f1ce96f-4b70-4b05-9e9a-d419f33e702e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61a4b34360f24208980ffb297e87d2e9'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:57.544493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1198/2391] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:02:57 2022] DELETE /identity/v3/users/fb7018c9b76d4f4ab74ed8e45d9e4bb5 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:02:57.549184 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f525ca-de9d-4206-aeb8-c1c22555e883 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:57.549477 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f525ca-de9d-4206-aeb8-c1c22555e883 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:57.549757 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f525ca-de9d-4206-aeb8-c1c22555e883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:57.569626 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1194/2392] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:02:57 2022] DELETE /identity/v3/users/61a4b34360f24208980ffb297e87d2e9 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:57.581319 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dcf5af-23a6-452b-8da9-cc50fcce7968 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:57.581908 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dcf5af-23a6-452b-8da9-cc50fcce7968 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:57.582294 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-77f525ca-de9d-4206-aeb8-c1c22555e883 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:57.582516 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dcf5af-23a6-452b-8da9-cc50fcce7968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:57.583586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-77f525ca-de9d-4206-aeb8-c1c22555e883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:57.609354 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-b3dcf5af-23a6-452b-8da9-cc50fcce7968 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:57.611347 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-b3dcf5af-23a6-452b-8da9-cc50fcce7968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:57.623351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:57.719695 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1195/2393] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:57.745444 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dcf5af-23a6-452b-8da9-cc50fcce7968 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:57.747004 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-b3dcf5af-23a6-452b-8da9-cc50fcce7968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:57.751596 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1199/2394] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:57.759273 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f525ca-de9d-4206-aeb8-c1c22555e883 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:57.761047 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-77f525ca-de9d-4206-aeb8-c1c22555e883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:57.782119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:57.834622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f525ca-de9d-4206-aeb8-c1c22555e883 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:57.838647 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b747cb-75f1-42f8-82ec-4a57fc52b952 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:57.839402 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b747cb-75f1-42f8-82ec-4a57fc52b952 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:57.839804 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b747cb-75f1-42f8-82ec-4a57fc52b952 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:57.840869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b747cb-75f1-42f8-82ec-4a57fc52b952 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:57.853759 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b747cb-75f1-42f8-82ec-4a57fc52b952 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:57.855276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b747cb-75f1-42f8-82ec-4a57fc52b952 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:57.879402 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dcf5af-23a6-452b-8da9-cc50fcce7968 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:57.880430 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1200/2395] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:02:57.892864 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e27c45-d93b-44b2-99e6-2fe6a0c6b603 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:57.894104 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e27c45-d93b-44b2-99e6-2fe6a0c6b603 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:57.894854 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e27c45-d93b-44b2-99e6-2fe6a0c6b603 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:57.895716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e27c45-d93b-44b2-99e6-2fe6a0c6b603 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:57.914241 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e27c45-d93b-44b2-99e6-2fe6a0c6b603 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:57.918299 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e27c45-d93b-44b2-99e6-2fe6a0c6b603 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:57.941316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1196/2396] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:02:58.512416 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b747cb-75f1-42f8-82ec-4a57fc52b952 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:58.514023 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f8b747cb-75f1-42f8-82ec-4a57fc52b952 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:58.517410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e27c45-d93b-44b2-99e6-2fe6a0c6b603 service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:58.520366 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-38e27c45-d93b-44b2-99e6-2fe6a0c6b603 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:58.549746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b747cb-75f1-42f8-82ec-4a57fc52b952 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:58.557022 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e7f31-3531-451f-ad05-14fcf30eb911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:58.557942 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e7f31-3531-451f-ad05-14fcf30eb911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:58.558862 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e7f31-3531-451f-ad05-14fcf30eb911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:58.559338 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e7f31-3531-451f-ad05-14fcf30eb911 admin admin] PATH_INFO: `/v3/projects/3ccf078eafe84a84ac66518c5ff4fc1e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:58.559810 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e27c45-d93b-44b2-99e6-2fe6a0c6b603 service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:58.561150 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e7f31-3531-451f-ad05-14fcf30eb911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ccf078eafe84a84ac66518c5ff4fc1e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:58.564529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e7f31-3531-451f-ad05-14fcf30eb911 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:58.600355 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3da81-c66b-4171-a31e-8adcb78b4dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:58.601024 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3da81-c66b-4171-a31e-8adcb78b4dd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:58.601313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3da81-c66b-4171-a31e-8adcb78b4dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:58.601596 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3da81-c66b-4171-a31e-8adcb78b4dd1 admin admin] PATH_INFO: `/v3/projects/0b1662fb7a414e3b9c9da636ad028e95` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:58.602790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d3da81-c66b-4171-a31e-8adcb78b4dd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b1662fb7a414e3b9c9da636ad028e95)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:58.604776 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d3da81-c66b-4171-a31e-8adcb78b4dd1 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:58.715392 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f5d3da81-c66b-4171-a31e-8adcb78b4dd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b1662fb7a414e3b9c9da636ad028e95', 'request_id': 'req-f5d3da81-c66b-4171-a31e-8adcb78b4dd1'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:58.733938 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f5d3da81-c66b-4171-a31e-8adcb78b4dd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b1662fb7a414e3b9c9da636ad028e95', 'request_id': 'req-f5d3da81-c66b-4171-a31e-8adcb78b4dd1'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:58.735457 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-867e7f31-3531-451f-ad05-14fcf30eb911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ccf078eafe84a84ac66518c5ff4fc1e', 'request_id': 'req-867e7f31-3531-451f-ad05-14fcf30eb911'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:58.744601 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f5d3da81-c66b-4171-a31e-8adcb78b4dd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b1662fb7a414e3b9c9da636ad028e95', 'request_id': 'req-f5d3da81-c66b-4171-a31e-8adcb78b4dd1'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:58.759935 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f5d3da81-c66b-4171-a31e-8adcb78b4dd1 admin admin] The token cache is being invalidate because project 0b1662fb7a414e3b9c9da636ad028e95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:58.760321 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f5d3da81-c66b-4171-a31e-8adcb78b4dd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:58.760905 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f5d3da81-c66b-4171-a31e-8adcb78b4dd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:58.762662 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1197/2397] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:58 2022] DELETE /identity/v3/projects/0b1662fb7a414e3b9c9da636ad028e95 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:58.767031 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-867e7f31-3531-451f-ad05-14fcf30eb911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ccf078eafe84a84ac66518c5ff4fc1e', 'request_id': 'req-867e7f31-3531-451f-ad05-14fcf30eb911'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:58.768180 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcd23d-25c7-444e-a6d2-925b0b1449f7 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:58.768475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcd23d-25c7-444e-a6d2-925b0b1449f7 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:58.768755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcd23d-25c7-444e-a6d2-925b0b1449f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:58.787190 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-867e7f31-3531-451f-ad05-14fcf30eb911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ccf078eafe84a84ac66518c5ff4fc1e', 'request_id': 'req-867e7f31-3531-451f-ad05-14fcf30eb911'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:58.805011 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-1afcd23d-25c7-444e-a6d2-925b0b1449f7 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:58.807083 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-1afcd23d-25c7-444e-a6d2-925b0b1449f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:58.818764 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-867e7f31-3531-451f-ad05-14fcf30eb911 admin admin] The token cache is being invalidate because project 3ccf078eafe84a84ac66518c5ff4fc1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:02:58.819348 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-867e7f31-3531-451f-ad05-14fcf30eb911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:58.820189 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-867e7f31-3531-451f-ad05-14fcf30eb911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:02:58.822317 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1201/2398] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:02:58 2022] DELETE /identity/v3/projects/3ccf078eafe84a84ac66518c5ff4fc1e => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:02:58.866939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:58.919406 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1198/2399] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:58.925385 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf691ea2-d249-4248-b9ab-751acd7d97a8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:58.926044 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf691ea2-d249-4248-b9ab-751acd7d97a8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:58.926555 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf691ea2-d249-4248-b9ab-751acd7d97a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:58.981056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-cf691ea2-d249-4248-b9ab-751acd7d97a8 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:58.984680 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cf691ea2-d249-4248-b9ab-751acd7d97a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:58.999209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1202/2400] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:59.000055 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afcd23d-25c7-444e-a6d2-925b0b1449f7 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:59.015954 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afcd23d-25c7-444e-a6d2-925b0b1449f7 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:59.044564 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab78e58-d429-47ce-8c3b-62d0679329ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:59.045304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab78e58-d429-47ce-8c3b-62d0679329ce admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:59.045570 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab78e58-d429-47ce-8c3b-62d0679329ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:59.045900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab78e58-d429-47ce-8c3b-62d0679329ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:59.046629 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab78e58-d429-47ce-8c3b-62d0679329ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:02:59.047107 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab78e58-d429-47ce-8c3b-62d0679329ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:59.051105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab78e58-d429-47ce-8c3b-62d0679329ce admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:59.087757 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1199/2401] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 05:02:59.089983 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf691ea2-d249-4248-b9ab-751acd7d97a8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:59.105793 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf691ea2-d249-4248-b9ab-751acd7d97a8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:59.130653 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de318b-918d-475b-9e71-bed9043646fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:59.133810 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de318b-918d-475b-9e71-bed9043646fe admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:59.136691 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de318b-918d-475b-9e71-bed9043646fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:59.137954 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de318b-918d-475b-9e71-bed9043646fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:59.148663 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7de318b-918d-475b-9e71-bed9043646fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:59.150759 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7de318b-918d-475b-9e71-bed9043646fe admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:59.224771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1203/2402] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:02:59 2022] POST /identity/v3/projects => generated 377 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:59.234313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab78e58-d429-47ce-8c3b-62d0679329ce admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:59.252672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab78e58-d429-47ce-8c3b-62d0679329ce admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:59.349619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afbb7c-db7b-48e1-b68b-d46b99e54203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:59.350268 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afbb7c-db7b-48e1-b68b-d46b99e54203 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:59.350539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afbb7c-db7b-48e1-b68b-d46b99e54203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:59.350809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afbb7c-db7b-48e1-b68b-d46b99e54203 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:59.351836 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9afbb7c-db7b-48e1-b68b-d46b99e54203 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:59.354005 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9afbb7c-db7b-48e1-b68b-d46b99e54203 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:59.406681 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1200/2403] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:02:59 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:02:59.409351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de318b-918d-475b-9e71-bed9043646fe admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:59.420603 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de318b-918d-475b-9e71-bed9043646fe admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:59.428401 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f7a42-0a16-48fe-ad6c-68934f9c4549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:59.429488 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f7a42-0a16-48fe-ad6c-68934f9c4549 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:59.429961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f7a42-0a16-48fe-ad6c-68934f9c4549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:59.430451 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f7a42-0a16-48fe-ad6c-68934f9c4549 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:59.431420 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7f7a42-0a16-48fe-ad6c-68934f9c4549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:59.434212 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7f7a42-0a16-48fe-ad6c-68934f9c4549 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:59.446911 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1204/2404] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:59.451472 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afbb7c-db7b-48e1-b68b-d46b99e54203 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:59.470496 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afbb7c-db7b-48e1-b68b-d46b99e54203 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:59.476034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc34bd7-00ba-461c-9d88-06af4d73bfd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:59.476765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc34bd7-00ba-461c-9d88-06af4d73bfd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:59.477112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc34bd7-00ba-461c-9d88-06af4d73bfd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:59.477483 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc34bd7-00ba-461c-9d88-06af4d73bfd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:59.482379 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc34bd7-00ba-461c-9d88-06af4d73bfd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:59.484573 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc34bd7-00ba-461c-9d88-06af4d73bfd2 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:59.503415 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1201/2405] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:02:59.504498 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f7a42-0a16-48fe-ad6c-68934f9c4549 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:59.521616 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f7a42-0a16-48fe-ad6c-68934f9c4549 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:59.527879 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8d340-126b-468f-b512-e2ef57dac9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:59.528641 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8d340-126b-468f-b512-e2ef57dac9c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:59.528926 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8d340-126b-468f-b512-e2ef57dac9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:59.529213 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8d340-126b-468f-b512-e2ef57dac9c1 admin admin] PATH_INFO: `/v3/projects/987757bb89e64419822e021b15a65c31/users/148ea5a9b7a74172b290696be99aa5c1/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:59.547957 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e8d340-126b-468f-b512-e2ef57dac9c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=987757bb89e64419822e021b15a65c31, user_id=148ea5a9b7a74172b290696be99aa5c1, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:59.550686 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e8d340-126b-468f-b512-e2ef57dac9c1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:59.575262 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1205/2406] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:02:59 2022] PUT /identity/v3/projects/987757bb89e64419822e021b15a65c31/users/148ea5a9b7a74172b290696be99aa5c1/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:02:59.578846 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73cabd-a2a1-4c99-8b1c-e52e99e09de1 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:59.579321 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73cabd-a2a1-4c99-8b1c-e52e99e09de1 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:59.579508 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73cabd-a2a1-4c99-8b1c-e52e99e09de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:59.603976 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-0d73cabd-a2a1-4c99-8b1c-e52e99e09de1 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:02:59.605556 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-0d73cabd-a2a1-4c99-8b1c-e52e99e09de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:59.742623 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1202/2407] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:02:59.754525 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8d340-126b-468f-b512-e2ef57dac9c1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:02:59.766234 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-38e8d340-126b-468f-b512-e2ef57dac9c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:02:59.836733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:02:59.860293 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8d340-126b-468f-b512-e2ef57dac9c1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:02:59.864158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67883c-6d69-4698-aaaf-dcbc2059d928 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:02:59.864811 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67883c-6d69-4698-aaaf-dcbc2059d928 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:02:59.865099 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67883c-6d69-4698-aaaf-dcbc2059d928 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:02:59.865378 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67883c-6d69-4698-aaaf-dcbc2059d928 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:02:59.880572 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a67883c-6d69-4698-aaaf-dcbc2059d928 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:02:59.882055 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a67883c-6d69-4698-aaaf-dcbc2059d928 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:02:59.896914 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1206/2408] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:02:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:03:01.296869 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976fbeb-3fc4-459d-bfc0-0a443497516d None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:01.297494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976fbeb-3fc4-459d-bfc0-0a443497516d None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:01.297494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976fbeb-3fc4-459d-bfc0-0a443497516d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:01.320237 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-d976fbeb-3fc4-459d-bfc0-0a443497516d None None] MFA Rules not processed for user `f0cc1af7d7d642889a1a11103f087e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:03:01.321940 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-d976fbeb-3fc4-459d-bfc0-0a443497516d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:03:01.385801 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:03:01.406351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1203/2409] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:03:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:03:04.711503 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67883c-6d69-4698-aaaf-dcbc2059d928 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:03:04.718221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4a67883c-6d69-4698-aaaf-dcbc2059d928 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:03:04.729595 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976fbeb-3fc4-459d-bfc0-0a443497516d None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:03:04.731976 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-d976fbeb-3fc4-459d-bfc0-0a443497516d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:03:04.860660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67883c-6d69-4698-aaaf-dcbc2059d928 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:03:04.867940 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4619b-9d61-4701-b663-b7f34fdb9449 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5049dba7ffde47b8af3e6fd55b5703e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:03:04.868779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4619b-9d61-4701-b663-b7f34fdb9449 service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:04.869232 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4619b-9d61-4701-b663-b7f34fdb9449 service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:04.869593 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4619b-9d61-4701-b663-b7f34fdb9449 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:04.872372 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-31a4619b-9d61-4701-b663-b7f34fdb9449 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:03:04.883654 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976fbeb-3fc4-459d-bfc0-0a443497516d None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:03:04.888515 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a4619b-9d61-4701-b663-b7f34fdb9449 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:03:04.889709 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1199e-f074-481d-8658-cc93599f9223 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5049dba7ffde47b8af3e6fd55b5703e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:03:04.890365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1199e-f074-481d-8658-cc93599f9223 service placement] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:04.890694 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a4619b-9d61-4701-b663-b7f34fdb9449 service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:03:04.890877 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1199e-f074-481d-8658-cc93599f9223 service placement] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:04.891042 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1199e-f074-481d-8658-cc93599f9223 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:04.912044 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1207/2410] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:03:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:03:04.912352 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1199e-f074-481d-8658-cc93599f9223 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:03:04.914565 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1199e-f074-481d-8658-cc93599f9223 service placement] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:03:04.943176 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1204/2411] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:03:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:03:05.411354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55062366-0b65-48c1-b1e2-3b99067f80c0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:05.411354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55062366-0b65-48c1-b1e2-3b99067f80c0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:05.412160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55062366-0b65-48c1-b1e2-3b99067f80c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:05.471970 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-55062366-0b65-48c1-b1e2-3b99067f80c0 None None] MFA Rules not processed for user `148ea5a9b7a74172b290696be99aa5c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:03:05.474120 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-55062366-0b65-48c1-b1e2-3b99067f80c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:03:05.607419 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1208/2412] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:03:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:03:05.615888 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1199e-f074-481d-8658-cc93599f9223 service placement] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:03:05.642574 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1199e-f074-481d-8658-cc93599f9223 service placement] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:03:05.665218 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828be31-40a7-419f-aaac-6a84f31f6801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:03:05.667134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828be31-40a7-419f-aaac-6a84f31f6801 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:05.667134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828be31-40a7-419f-aaac-6a84f31f6801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:05.667134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828be31-40a7-419f-aaac-6a84f31f6801 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:05.667506 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828be31-40a7-419f-aaac-6a84f31f6801 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:03:05.669895 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828be31-40a7-419f-aaac-6a84f31f6801 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:03:05.714104 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1205/2413] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:03:05 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:03:05.719933 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55062366-0b65-48c1-b1e2-3b99067f80c0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:03:05.735901 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55062366-0b65-48c1-b1e2-3b99067f80c0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:03:05.879215 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec2013-7771-4965-a0cf-2ffe88b25eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:03:05.880242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec2013-7771-4965-a0cf-2ffe88b25eda admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:05.880700 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec2013-7771-4965-a0cf-2ffe88b25eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:05.881135 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec2013-7771-4965-a0cf-2ffe88b25eda admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:05.882077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec2013-7771-4965-a0cf-2ffe88b25eda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:03:05.884309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec2013-7771-4965-a0cf-2ffe88b25eda admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:03:05.893240 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:03:05.950604 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1209/2414] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:03:05 2022] POST /identity/v3/users => generated 296 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:03:05.954765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828be31-40a7-419f-aaac-6a84f31f6801 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:03:05.982747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828be31-40a7-419f-aaac-6a84f31f6801 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:03:05.996293 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547bedf7-7b0b-4b37-83ce-0fcf07da0b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:03:05.997349 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bedf7-7b0b-4b37-83ce-0fcf07da0b01 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:05.997850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bedf7-7b0b-4b37-83ce-0fcf07da0b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:05.998281 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bedf7-7b0b-4b37-83ce-0fcf07da0b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:05.999238 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547bedf7-7b0b-4b37-83ce-0fcf07da0b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:03:06.001165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547bedf7-7b0b-4b37-83ce-0fcf07da0b01 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:03:06.032549 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1206/2415] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:03:06.035912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec2013-7771-4965-a0cf-2ffe88b25eda admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:03:06.047470 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec2013-7771-4965-a0cf-2ffe88b25eda admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:03:06.053572 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75385a4a-2a0b-436f-aa5d-76965a72d507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:03:06.054509 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75385a4a-2a0b-436f-aa5d-76965a72d507 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:06.054797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75385a4a-2a0b-436f-aa5d-76965a72d507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:06.055087 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75385a4a-2a0b-436f-aa5d-76965a72d507 admin admin] PATH_INFO: `/v3/projects/6874e6654d734910b67db92475ece16e/users/c9817b6598dc480898e0b9f5c9ee5e5d/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:06.076917 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75385a4a-2a0b-436f-aa5d-76965a72d507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6874e6654d734910b67db92475ece16e, user_id=c9817b6598dc480898e0b9f5c9ee5e5d, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:03:06.080032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75385a4a-2a0b-436f-aa5d-76965a72d507 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:03:06.106200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1210/2416] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:03:06 2022] PUT /identity/v3/projects/6874e6654d734910b67db92475ece16e/users/c9817b6598dc480898e0b9f5c9ee5e5d/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:03:06.113831 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547bedf7-7b0b-4b37-83ce-0fcf07da0b01 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:03:06.149747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547bedf7-7b0b-4b37-83ce-0fcf07da0b01 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:03:06.273300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957022a-15c1-4bf2-b8a5-026b3f9319ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:03:06.273955 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957022a-15c1-4bf2-b8a5-026b3f9319ed admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:06.274243 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957022a-15c1-4bf2-b8a5-026b3f9319ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:06.274523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957022a-15c1-4bf2-b8a5-026b3f9319ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:06.275246 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957022a-15c1-4bf2-b8a5-026b3f9319ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:03:06.277083 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957022a-15c1-4bf2-b8a5-026b3f9319ed admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:03:06.299677 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1207/2417] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:03:06.304384 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75385a4a-2a0b-436f-aa5d-76965a72d507 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:03:06.328918 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75385a4a-2a0b-436f-aa5d-76965a72d507 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:03:06.343116 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c1ccea-d4b5-40cf-98bd-b94ce0f72a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:03:06.344145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1ccea-d4b5-40cf-98bd-b94ce0f72a19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:06.344458 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1ccea-d4b5-40cf-98bd-b94ce0f72a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:06.344786 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1ccea-d4b5-40cf-98bd-b94ce0f72a19 admin admin] PATH_INFO: `/v3/projects/6874e6654d734910b67db92475ece16e/users/c9817b6598dc480898e0b9f5c9ee5e5d/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:06.349177 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c1ccea-d4b5-40cf-98bd-b94ce0f72a19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6874e6654d734910b67db92475ece16e, user_id=c9817b6598dc480898e0b9f5c9ee5e5d, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:03:06.356320 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c1ccea-d4b5-40cf-98bd-b94ce0f72a19 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:03:06.381803 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1211/2418] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:03:06 2022] PUT /identity/v3/projects/6874e6654d734910b67db92475ece16e/users/c9817b6598dc480898e0b9f5c9ee5e5d/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:03:06.870609 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc98762-3f5c-428f-b921-cd66e3184bf8 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:06.871122 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc98762-3f5c-428f-b921-cd66e3184bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:06.871758 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc98762-3f5c-428f-b921-cd66e3184bf8 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:06.874058 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1208/2419] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jul 26 05:03:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 05:03:06.882373 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56005221-0c05-4666-9e59-95950da16af0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:06.882831 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56005221-0c05-4666-9e59-95950da16af0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:06.883513 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56005221-0c05-4666-9e59-95950da16af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:06.912204 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-56005221-0c05-4666-9e59-95950da16af0 None None] MFA Rules not processed for user `2faeed9002ee4154bc980c146cd06bcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:03:06.913480 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-56005221-0c05-4666-9e59-95950da16af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:03:07.013536 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1212/2420] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jul 26 05:03:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:03:07.040439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc98762-3f5c-428f-b921-cd66e3184bf8 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:03:07.042688 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-4fc98762-3f5c-428f-b921-cd66e3184bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:03:07.179912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:03:07.204301 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc98762-3f5c-428f-b921-cd66e3184bf8 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:03:07.215321 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8192f5-6a18-411f-8466-c1fcf9b90a77 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6946a788d2ef49ff873ca8e81990527e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:03:07.216330 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8192f5-6a18-411f-8466-c1fcf9b90a77 service glance] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:07.216692 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8192f5-6a18-411f-8466-c1fcf9b90a77 service glance] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:07.216973 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8192f5-6a18-411f-8466-c1fcf9b90a77 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:07.232618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8192f5-6a18-411f-8466-c1fcf9b90a77 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:03:07.234653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8192f5-6a18-411f-8466-c1fcf9b90a77 service glance] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:03:07.252504 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1209/2421] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:03:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:03:07.383880 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60528877-7279-4c25-9998-f9af006b5db7 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:07.384285 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60528877-7279-4c25-9998-f9af006b5db7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:07.390747 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60528877-7279-4c25-9998-f9af006b5db7 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:07.392925 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1213/2422] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jul 26 05:03:07 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 05:03:07.398790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880deda-6766-48a2-9bb8-13ea9df5be04 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:07.400030 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880deda-6766-48a2-9bb8-13ea9df5be04 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:07.400519 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880deda-6766-48a2-9bb8-13ea9df5be04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:07.435254 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-6880deda-6766-48a2-9bb8-13ea9df5be04 None None] MFA Rules not processed for user `dbdb8997891241e9b2e829feb8023f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:03:07.437599 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-6880deda-6766-48a2-9bb8-13ea9df5be04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:03:07.540208 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1210/2423] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jul 26 05:03:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:03:07.562036 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60528877-7279-4c25-9998-f9af006b5db7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:03:07.566599 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-60528877-7279-4c25-9998-f9af006b5db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:03:07.599607 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60528877-7279-4c25-9998-f9af006b5db7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:03:07.615054 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1269296-6933-4c90-969f-7358bb61ea8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:03:07.617058 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1269296-6933-4c90-969f-7358bb61ea8b service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:07.620474 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1269296-6933-4c90-969f-7358bb61ea8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:07.620939 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1269296-6933-4c90-969f-7358bb61ea8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:07.646980 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1269296-6933-4c90-969f-7358bb61ea8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:03:07.650946 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1269296-6933-4c90-969f-7358bb61ea8b service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:03:07.681086 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1214/2424] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:03:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:03:07.847638 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629c6d7-6e91-4756-9a8d-feee2d69f9b7 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:07.848244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629c6d7-6e91-4756-9a8d-feee2d69f9b7 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:07.848777 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629c6d7-6e91-4756-9a8d-feee2d69f9b7 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:07.850372 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1211/2425] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jul 26 05:03:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 05:03:07.857088 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2c17a-4f96-41be-8b4b-80492438ae28 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:07.857958 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2c17a-4f96-41be-8b4b-80492438ae28 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:07.858348 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2c17a-4f96-41be-8b4b-80492438ae28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:07.905378 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6ad2c17a-4f96-41be-8b4b-80492438ae28 None None] MFA Rules not processed for user `2faeed9002ee4154bc980c146cd06bcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:03:07.909907 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6ad2c17a-4f96-41be-8b4b-80492438ae28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:03:07.938779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1215/2426] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jul 26 05:03:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:03:07.972700 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629c6d7-6e91-4756-9a8d-feee2d69f9b7 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:03:08.001581 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629c6d7-6e91-4756-9a8d-feee2d69f9b7 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:03:08.006819 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf52d1-4174-4df6-b8ac-b61cb75477e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:03:08.007841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf52d1-4174-4df6-b8ac-b61cb75477e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:08.008408 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf52d1-4174-4df6-b8ac-b61cb75477e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:08.009686 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf52d1-4174-4df6-b8ac-b61cb75477e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:08.035987 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cf52d1-4174-4df6-b8ac-b61cb75477e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:03:08.038350 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cf52d1-4174-4df6-b8ac-b61cb75477e9 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:03:08.057907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1212/2427] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:03:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:03:12.645287 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2c17a-4f96-41be-8b4b-80492438ae28 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:03:12.646859 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6ad2c17a-4f96-41be-8b4b-80492438ae28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:03:12.778418 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2c17a-4f96-41be-8b4b-80492438ae28 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:03:12.786056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8a5e2-7d7f-40a1-8376-321e1209c648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:03:12.786861 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8a5e2-7d7f-40a1-8376-321e1209c648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:12.787395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8a5e2-7d7f-40a1-8376-321e1209c648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:12.788100 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8a5e2-7d7f-40a1-8376-321e1209c648 admin admin] PATH_INFO: `/v3/users/fa8e1ff7b31d47b29c7fe11b0eb9dce4` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:12.794363 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d8a5e2-7d7f-40a1-8376-321e1209c648 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa8e1ff7b31d47b29c7fe11b0eb9dce4)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:03:12.795800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d8a5e2-7d7f-40a1-8376-321e1209c648 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:03:12.875417 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:03:12.924726 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-89d8a5e2-7d7f-40a1-8376-321e1209c648 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa8e1ff7b31d47b29c7fe11b0eb9dce4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:03:12.942019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-89d8a5e2-7d7f-40a1-8376-321e1209c648 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa8e1ff7b31d47b29c7fe11b0eb9dce4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:03:12.942847 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-89d8a5e2-7d7f-40a1-8376-321e1209c648 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa8e1ff7b31d47b29c7fe11b0eb9dce4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:03:12.959208 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-89d8a5e2-7d7f-40a1-8376-321e1209c648 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa8e1ff7b31d47b29c7fe11b0eb9dce4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:03:12.960268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-89d8a5e2-7d7f-40a1-8376-321e1209c648 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa8e1ff7b31d47b29c7fe11b0eb9dce4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:03:12.997729 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1216/2428] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:03:12 2022] DELETE /identity/v3/users/fa8e1ff7b31d47b29c7fe11b0eb9dce4 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:03:13.002069 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c382a54-a83b-43f6-8ad3-b9f15b16d71a None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:13.003832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c382a54-a83b-43f6-8ad3-b9f15b16d71a None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:13.004620 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c382a54-a83b-43f6-8ad3-b9f15b16d71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:13.032077 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-8c382a54-a83b-43f6-8ad3-b9f15b16d71a None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:03:13.034403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-8c382a54-a83b-43f6-8ad3-b9f15b16d71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:03:13.155109 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1213/2429] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:03:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:03:13.165343 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8a5e2-7d7f-40a1-8376-321e1209c648 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:03:13.167837 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-89d8a5e2-7d7f-40a1-8376-321e1209c648 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:03:13.260449 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8a5e2-7d7f-40a1-8376-321e1209c648 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:03:13.265364 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7597d9-8d25-42d1-9e23-969c443a5031 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:03:13.266071 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7597d9-8d25-42d1-9e23-969c443a5031 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:13.266356 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7597d9-8d25-42d1-9e23-969c443a5031 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:13.266627 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7597d9-8d25-42d1-9e23-969c443a5031 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:13.277923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7597d9-8d25-42d1-9e23-969c443a5031 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:03:13.279921 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7597d9-8d25-42d1-9e23-969c443a5031 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:03:13.296032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1217/2430] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:03:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:03:13.635245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bcbe2-35b7-4bea-863d-c4b8beaadecf None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:13.635867 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bcbe2-35b7-4bea-863d-c4b8beaadecf None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:13.636295 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bcbe2-35b7-4bea-863d-c4b8beaadecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:13.673064 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-1b1bcbe2-35b7-4bea-863d-c4b8beaadecf None None] MFA Rules not processed for user `3a8f3eb811ae40ad9cb64261f95a992e`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:03:13.674911 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-1b1bcbe2-35b7-4bea-863d-c4b8beaadecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:03:13.821839 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1214/2431] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jul 26 05:03:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:03:13.835844 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7597d9-8d25-42d1-9e23-969c443a5031 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:03:13.840866 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fc7597d9-8d25-42d1-9e23-969c443a5031 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:03:13.882608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1bcbe2-35b7-4bea-863d-c4b8beaadecf None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:03:13.884848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-1b1bcbe2-35b7-4bea-863d-c4b8beaadecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:03:13.943820 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1bcbe2-35b7-4bea-863d-c4b8beaadecf None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:03:13.957253 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9b463d-5d78-42c1-973e-4f1cdf2ac0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:03:13.957898 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b463d-5d78-42c1-973e-4f1cdf2ac0b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:13.959051 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b463d-5d78-42c1-973e-4f1cdf2ac0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:13.959168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b463d-5d78-42c1-973e-4f1cdf2ac0b3 admin admin] PATH_INFO: `/v3/projects/a049ed8c11ae45ae9ed14757f1e00b00` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:13.967019 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9b463d-5d78-42c1-973e-4f1cdf2ac0b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a049ed8c11ae45ae9ed14757f1e00b00)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:03:13.983536 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9b463d-5d78-42c1-973e-4f1cdf2ac0b3 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:03:14.047737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:03:14.089055 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7597d9-8d25-42d1-9e23-969c443a5031 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:03:14.099352 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8f9b463d-5d78-42c1-973e-4f1cdf2ac0b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a049ed8c11ae45ae9ed14757f1e00b00', 'request_id': 'req-8f9b463d-5d78-42c1-973e-4f1cdf2ac0b3'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:03:14.111391 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8f9b463d-5d78-42c1-973e-4f1cdf2ac0b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a049ed8c11ae45ae9ed14757f1e00b00', 'request_id': 'req-8f9b463d-5d78-42c1-973e-4f1cdf2ac0b3'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:03:14.121567 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8f9b463d-5d78-42c1-973e-4f1cdf2ac0b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a049ed8c11ae45ae9ed14757f1e00b00', 'request_id': 'req-8f9b463d-5d78-42c1-973e-4f1cdf2ac0b3'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:03:14.149371 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8f9b463d-5d78-42c1-973e-4f1cdf2ac0b3 admin admin] The token cache is being invalidate because project a049ed8c11ae45ae9ed14757f1e00b00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:03:14.151051 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8f9b463d-5d78-42c1-973e-4f1cdf2ac0b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:03:14.152189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8f9b463d-5d78-42c1-973e-4f1cdf2ac0b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:03:14.154364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1215/2432] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:03:13 2022] DELETE /identity/v3/projects/a049ed8c11ae45ae9ed14757f1e00b00 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:03:14.167686 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:03:14.212444 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09160e8e-de33-4480-bb2e-05b29e27f220 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5218319aaa5424882714bffcbeee730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:03:14.214832 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09160e8e-de33-4480-bb2e-05b29e27f220 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:14.215362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09160e8e-de33-4480-bb2e-05b29e27f220 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:14.216060 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09160e8e-de33-4480-bb2e-05b29e27f220 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:14.218547 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-09160e8e-de33-4480-bb2e-05b29e27f220 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:03:14.381574 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09160e8e-de33-4480-bb2e-05b29e27f220 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:03:14.394219 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09160e8e-de33-4480-bb2e-05b29e27f220 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:03:14.410974 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1218/2433] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jul 26 05:03:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 05:03:14.663077 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac3048-a591-4476-b0a7-e14b18c36b49 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:14.663434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac3048-a591-4476-b0a7-e14b18c36b49 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:14.663746 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac3048-a591-4476-b0a7-e14b18c36b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:14.722947 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-73ac3048-a591-4476-b0a7-e14b18c36b49 None None] MFA Rules not processed for user `c9817b6598dc480898e0b9f5c9ee5e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:03:14.724403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-73ac3048-a591-4476-b0a7-e14b18c36b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:03:14.891141 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1216/2434] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:03:14 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:03:14.898077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09160e8e-de33-4480-bb2e-05b29e27f220 service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:03:14.899836 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-09160e8e-de33-4480-bb2e-05b29e27f220 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:03:15.047519 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09160e8e-de33-4480-bb2e-05b29e27f220 service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:03:15.057982 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258859e-8f41-40bd-89b6-a1690274df42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:03:15.060354 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258859e-8f41-40bd-89b6-a1690274df42 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:15.060651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258859e-8f41-40bd-89b6-a1690274df42 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:15.060934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258859e-8f41-40bd-89b6-a1690274df42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:15.069345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e258859e-8f41-40bd-89b6-a1690274df42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:03:15.243225 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e258859e-8f41-40bd-89b6-a1690274df42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:03:15.244894 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e258859e-8f41-40bd-89b6-a1690274df42 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:03:15.256513 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1219/2435] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:03:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:03:24.870163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:DELETE=6,keystone:SELECT=48,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:03:25.250943 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=88 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:03:34.631843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba4ef7-5c2b-4602-a9b1-c7557887b8fe None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:34.632388 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba4ef7-5c2b-4602-a9b1-c7557887b8fe None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:34.632388 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba4ef7-5c2b-4602-a9b1-c7557887b8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:34.656359 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-c7ba4ef7-5c2b-4602-a9b1-c7557887b8fe None None] MFA Rules not processed for user `148ea5a9b7a74172b290696be99aa5c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:03:34.657641 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c7ba4ef7-5c2b-4602-a9b1-c7557887b8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:03:34.666143 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1217/2436] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:03:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:03:38.387107 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78236daf-a3ae-4897-bbfb-7a61da1a0c08 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:38.387640 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78236daf-a3ae-4897-bbfb-7a61da1a0c08 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:38.388078 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78236daf-a3ae-4897-bbfb-7a61da1a0c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:38.414830 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-78236daf-a3ae-4897-bbfb-7a61da1a0c08 None None] MFA Rules not processed for user `3a8f3eb811ae40ad9cb64261f95a992e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:03:38.416230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-78236daf-a3ae-4897-bbfb-7a61da1a0c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:03:38.423990 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1220/2437] 10.222.0.17 () {58 vars in 967 bytes} [Tue Jul 26 05:03:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:03:38.437222 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba4ef7-5c2b-4602-a9b1-c7557887b8fe None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:03:38.438628 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c7ba4ef7-5c2b-4602-a9b1-c7557887b8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:03:38.453993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba4ef7-5c2b-4602-a9b1-c7557887b8fe None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:03:38.458862 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42070f2f-b99f-4fc0-b8ee-beb0ae7eac11 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5218319aaa5424882714bffcbeee730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:03:38.459492 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42070f2f-b99f-4fc0-b8ee-beb0ae7eac11 service swift] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:38.459811 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42070f2f-b99f-4fc0-b8ee-beb0ae7eac11 service swift] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:38.460082 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42070f2f-b99f-4fc0-b8ee-beb0ae7eac11 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:38.468950 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42070f2f-b99f-4fc0-b8ee-beb0ae7eac11 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:03:38.470917 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42070f2f-b99f-4fc0-b8ee-beb0ae7eac11 service swift] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:03:38.479037 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1218/2438] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jul 26 05:03:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 05:03:48.403918 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:03:48.475964 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=9 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:03:52.524514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78236daf-a3ae-4897-bbfb-7a61da1a0c08 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:03:52.526099 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-78236daf-a3ae-4897-bbfb-7a61da1a0c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:03:52.589220 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78236daf-a3ae-4897-bbfb-7a61da1a0c08 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:03:52.593101 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a1f52-deff-419e-b64d-ebbc0d126867 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:03:52.593747 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a1f52-deff-419e-b64d-ebbc0d126867 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:03:52.594026 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a1f52-deff-419e-b64d-ebbc0d126867 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:03:52.594301 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a1f52-deff-419e-b64d-ebbc0d126867 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:03:52.603715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a1f52-deff-419e-b64d-ebbc0d126867 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:03:52.605168 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a1f52-deff-419e-b64d-ebbc0d126867 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:03:52.614872 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1221/2439] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:03:52 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:04:02.610575 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=26 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:04:30.759316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc7cf3-03ae-4512-8907-5cf6f53727aa None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:04:30.759927 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc7cf3-03ae-4512-8907-5cf6f53727aa None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:04:30.759927 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc7cf3-03ae-4512-8907-5cf6f53727aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:04:30.779393 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-21cc7cf3-03ae-4512-8907-5cf6f53727aa None None] MFA Rules not processed for user `f996bed3f6064c6dbf04d26c7c04df7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:04:30.780766 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-21cc7cf3-03ae-4512-8907-5cf6f53727aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:04:30.848634 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1219/2440] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:04:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:04:35.787371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8169cad2-c1cc-4ac6-a093-a5552cb0b6ea None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:04:35.788044 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8169cad2-c1cc-4ac6-a093-a5552cb0b6ea None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:04:35.788157 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8169cad2-c1cc-4ac6-a093-a5552cb0b6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:04:35.809572 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8169cad2-c1cc-4ac6-a093-a5552cb0b6ea None None] MFA Rules not processed for user `f0cc1af7d7d642889a1a11103f087e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:04:35.810938 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8169cad2-c1cc-4ac6-a093-a5552cb0b6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:04:35.887609 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1222/2441] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:04:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:04:40.840789 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=25 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:04:45.878790 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=25 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:04:48.970359 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c4e1e-9182-4e44-a111-d76e3acd7e92 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:04:48.970917 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c4e1e-9182-4e44-a111-d76e3acd7e92 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:04:48.970917 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c4e1e-9182-4e44-a111-d76e3acd7e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:04:48.996173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-167c4e1e-9182-4e44-a111-d76e3acd7e92 None None] MFA Rules not processed for user `39de5897bd2d4e2b91c05d8dda5827be`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:04:48.997401 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-167c4e1e-9182-4e44-a111-d76e3acd7e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:04:49.063121 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1220/2442] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:04:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:04:54.724471 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8169cad2-c1cc-4ac6-a093-a5552cb0b6ea None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:04:54.735003 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8169cad2-c1cc-4ac6-a093-a5552cb0b6ea None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:04:54.739107 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43159bd6-85e8-4815-ad06-b32d46ec5e18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:04:54.740110 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43159bd6-85e8-4815-ad06-b32d46ec5e18 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:04:54.740363 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43159bd6-85e8-4815-ad06-b32d46ec5e18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:04:54.740762 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43159bd6-85e8-4815-ad06-b32d46ec5e18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:04:54.742954 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-43159bd6-85e8-4815-ad06-b32d46ec5e18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:04:54.757801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43159bd6-85e8-4815-ad06-b32d46ec5e18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:04:54.760485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43159bd6-85e8-4815-ad06-b32d46ec5e18 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:04:54.772775 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1223/2443] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:04:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:04:55.900493 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c4e1e-9182-4e44-a111-d76e3acd7e92 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:04:55.910868 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c4e1e-9182-4e44-a111-d76e3acd7e92 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:04:55.914651 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cdebd-5da0-45a1-872e-be537155b8f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:04:55.915427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cdebd-5da0-45a1-872e-be537155b8f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:04:55.915896 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cdebd-5da0-45a1-872e-be537155b8f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:04:55.916266 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cdebd-5da0-45a1-872e-be537155b8f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:04:55.918198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-8c8cdebd-5da0-45a1-872e-be537155b8f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:04:55.932234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8cdebd-5da0-45a1-872e-be537155b8f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:04:55.933900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8cdebd-5da0-45a1-872e-be537155b8f5 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:04:55.946460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1221/2444] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:04:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:05:04.767097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:05.941024 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=31 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:21.892625 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43159bd6-85e8-4815-ad06-b32d46ec5e18 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:21.894405 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-43159bd6-85e8-4815-ad06-b32d46ec5e18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:21.908950 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43159bd6-85e8-4815-ad06-b32d46ec5e18 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:21.916825 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ccd74c-f995-4bd3-b50b-c5967f95801f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:21.917730 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ccd74c-f995-4bd3-b50b-c5967f95801f service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:21.918132 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ccd74c-f995-4bd3-b50b-c5967f95801f service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:21.918692 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ccd74c-f995-4bd3-b50b-c5967f95801f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:21.921898 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-66ccd74c-f995-4bd3-b50b-c5967f95801f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:21.934500 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ccd74c-f995-4bd3-b50b-c5967f95801f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:21.936189 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ccd74c-f995-4bd3-b50b-c5967f95801f service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:21.946545 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1224/2445] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:05:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:05:26.805092 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cdebd-5da0-45a1-872e-be537155b8f5 service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:26.814781 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cdebd-5da0-45a1-872e-be537155b8f5 service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:26.819303 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f05ff55-6b32-4254-96f9-dca6d90685e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:26.819962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05ff55-6b32-4254-96f9-dca6d90685e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:26.820228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05ff55-6b32-4254-96f9-dca6d90685e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:26.820514 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05ff55-6b32-4254-96f9-dca6d90685e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:26.822771 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-4f05ff55-6b32-4254-96f9-dca6d90685e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:26.835218 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f05ff55-6b32-4254-96f9-dca6d90685e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:26.836701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f05ff55-6b32-4254-96f9-dca6d90685e1 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:26.848725 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1222/2446] 10.222.0.17 () {60 vars in 1396 bytes} [Tue Jul 26 05:05:26 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3258 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:05:31.941977 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:35.800470 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ccd74c-f995-4bd3-b50b-c5967f95801f service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:35.810807 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ccd74c-f995-4bd3-b50b-c5967f95801f service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:35.815420 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346adb2-aa39-42d8-88d1-be50c04cb325 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:35.816101 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346adb2-aa39-42d8-88d1-be50c04cb325 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:35.816412 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346adb2-aa39-42d8-88d1-be50c04cb325 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:35.816729 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346adb2-aa39-42d8-88d1-be50c04cb325 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:35.818545 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7346adb2-aa39-42d8-88d1-be50c04cb325 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:35.887054 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346adb2-aa39-42d8-88d1-be50c04cb325 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:35.888585 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346adb2-aa39-42d8-88d1-be50c04cb325 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:35.898620 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1225/2447] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:05:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:05:36.842102 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=6 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:39.292216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f05ff55-6b32-4254-96f9-dca6d90685e1 service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:39.293816 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-4f05ff55-6b32-4254-96f9-dca6d90685e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:39.381770 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f05ff55-6b32-4254-96f9-dca6d90685e1 service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:39.390613 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d80297-e158-4559-b831-e5ff7b87f80c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5049dba7ffde47b8af3e6fd55b5703e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:39.391320 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d80297-e158-4559-b831-e5ff7b87f80c service placement] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:39.397979 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d80297-e158-4559-b831-e5ff7b87f80c service placement] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:39.397979 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d80297-e158-4559-b831-e5ff7b87f80c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:39.397979 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-82d80297-e158-4559-b831-e5ff7b87f80c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:39.407172 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d80297-e158-4559-b831-e5ff7b87f80c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:39.408850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d80297-e158-4559-b831-e5ff7b87f80c service placement] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:39.419938 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1223/2448] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:05:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:05:39.574304 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346adb2-aa39-42d8-88d1-be50c04cb325 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:39.576584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7346adb2-aa39-42d8-88d1-be50c04cb325 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:39.589959 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346adb2-aa39-42d8-88d1-be50c04cb325 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:39.595141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bfef89-8ea7-429f-b0b5-a62c4214c7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:39.595839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bfef89-8ea7-429f-b0b5-a62c4214c7ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:39.596124 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bfef89-8ea7-429f-b0b5-a62c4214c7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:39.596409 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bfef89-8ea7-429f-b0b5-a62c4214c7ec admin admin] PATH_INFO: `/v3/users/f0cc1af7d7d642889a1a11103f087e79` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:39.597693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bfef89-8ea7-429f-b0b5-a62c4214c7ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0cc1af7d7d642889a1a11103f087e79)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:39.599684 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bfef89-8ea7-429f-b0b5-a62c4214c7ec admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:39.651813 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-24bfef89-8ea7-429f-b0b5-a62c4214c7ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0cc1af7d7d642889a1a11103f087e79'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:39.659585 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-24bfef89-8ea7-429f-b0b5-a62c4214c7ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0cc1af7d7d642889a1a11103f087e79'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:39.660355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-24bfef89-8ea7-429f-b0b5-a62c4214c7ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0cc1af7d7d642889a1a11103f087e79'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:39.672369 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-24bfef89-8ea7-429f-b0b5-a62c4214c7ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0cc1af7d7d642889a1a11103f087e79'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:39.673555 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-24bfef89-8ea7-429f-b0b5-a62c4214c7ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0cc1af7d7d642889a1a11103f087e79'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:39.692884 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1226/2449] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:05:39 2022] DELETE /identity/v3/users/f0cc1af7d7d642889a1a11103f087e79 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:05:39.869255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d80297-e158-4559-b831-e5ff7b87f80c service placement] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:39.882147 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d80297-e158-4559-b831-e5ff7b87f80c service placement] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:39.943797 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266575b8-97a9-48fc-9dc7-2e4d500ffa3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:39.943915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266575b8-97a9-48fc-9dc7-2e4d500ffa3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:39.944208 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266575b8-97a9-48fc-9dc7-2e4d500ffa3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:39.944483 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266575b8-97a9-48fc-9dc7-2e4d500ffa3d admin admin] PATH_INFO: `/v3/projects/1c772e5c90114978ad92818d58ca11c2` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:39.945717 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266575b8-97a9-48fc-9dc7-2e4d500ffa3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c772e5c90114978ad92818d58ca11c2)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:39.947719 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266575b8-97a9-48fc-9dc7-2e4d500ffa3d admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:39.997613 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-266575b8-97a9-48fc-9dc7-2e4d500ffa3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c772e5c90114978ad92818d58ca11c2', 'request_id': 'req-266575b8-97a9-48fc-9dc7-2e4d500ffa3d'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:40.007263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-266575b8-97a9-48fc-9dc7-2e4d500ffa3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c772e5c90114978ad92818d58ca11c2', 'request_id': 'req-266575b8-97a9-48fc-9dc7-2e4d500ffa3d'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:40.014731 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-266575b8-97a9-48fc-9dc7-2e4d500ffa3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c772e5c90114978ad92818d58ca11c2', 'request_id': 'req-266575b8-97a9-48fc-9dc7-2e4d500ffa3d'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:40.026731 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-266575b8-97a9-48fc-9dc7-2e4d500ffa3d admin admin] The token cache is being invalidate because project 1c772e5c90114978ad92818d58ca11c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:05:40.026971 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-266575b8-97a9-48fc-9dc7-2e4d500ffa3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:40.027717 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-266575b8-97a9-48fc-9dc7-2e4d500ffa3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:40.029255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1224/2450] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:05:39 2022] DELETE /identity/v3/projects/1c772e5c90114978ad92818d58ca11c2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:05:40.040320 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427f0e1-4baa-4223-9526-77f82c8d791e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:40.040623 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427f0e1-4baa-4223-9526-77f82c8d791e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:40.040915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427f0e1-4baa-4223-9526-77f82c8d791e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:40.062089 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4427f0e1-4baa-4223-9526-77f82c8d791e None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:05:40.063331 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4427f0e1-4baa-4223-9526-77f82c8d791e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:40.134763 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1227/2451] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:05:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:05:40.147694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffce8b3-43dc-424c-b4b3-7fe6b10cd1ea None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:40.147987 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffce8b3-43dc-424c-b4b3-7fe6b10cd1ea None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:40.148269 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffce8b3-43dc-424c-b4b3-7fe6b10cd1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:40.170264 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-bffce8b3-43dc-424c-b4b3-7fe6b10cd1ea None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:05:40.171513 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-bffce8b3-43dc-424c-b4b3-7fe6b10cd1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:40.180110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1225/2452] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:05:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:05:40.187616 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427f0e1-4baa-4223-9526-77f82c8d791e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:40.199814 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427f0e1-4baa-4223-9526-77f82c8d791e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:40.208655 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7d870-6f77-4017-84b4-32c4e8a0f91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:40.209582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7d870-6f77-4017-84b4-32c4e8a0f91d admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:40.209949 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7d870-6f77-4017-84b4-32c4e8a0f91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:40.210235 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7d870-6f77-4017-84b4-32c4e8a0f91d admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:40.211217 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7d870-6f77-4017-84b4-32c4e8a0f91d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:05:40.211582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7d870-6f77-4017-84b4-32c4e8a0f91d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:40.213890 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7d870-6f77-4017-84b4-32c4e8a0f91d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:40.215696 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:40.230126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1228/2453] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:05:40.234958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffce8b3-43dc-424c-b4b3-7fe6b10cd1ea None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:40.245471 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffce8b3-43dc-424c-b4b3-7fe6b10cd1ea None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:40.251153 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ca634-0662-4bc8-b135-c26ae3fee8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:40.251848 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ca634-0662-4bc8-b135-c26ae3fee8c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:40.252149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ca634-0662-4bc8-b135-c26ae3fee8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:40.252418 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ca634-0662-4bc8-b135-c26ae3fee8c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:40.253095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ca634-0662-4bc8-b135-c26ae3fee8c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:40.255215 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ca634-0662-4bc8-b135-c26ae3fee8c5 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:40.267721 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:40.288565 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1226/2454] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:05:40 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:05:40.293327 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7d870-6f77-4017-84b4-32c4e8a0f91d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:40.303884 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7d870-6f77-4017-84b4-32c4e8a0f91d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:40.366552 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fcc39c-f863-46b9-a97c-33d5e30ab7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:40.367181 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcc39c-f863-46b9-a97c-33d5e30ab7f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:40.367460 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcc39c-f863-46b9-a97c-33d5e30ab7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:40.367755 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcc39c-f863-46b9-a97c-33d5e30ab7f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:40.368435 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fcc39c-f863-46b9-a97c-33d5e30ab7f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:40.370425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fcc39c-f863-46b9-a97c-33d5e30ab7f3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:40.399778 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1229/2455] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:05:40 2022] POST /identity/v3/users => generated 282 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:05:40.405408 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ca634-0662-4bc8-b135-c26ae3fee8c5 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:40.419464 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ca634-0662-4bc8-b135-c26ae3fee8c5 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:40.424393 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea0040-b2df-4db5-a697-2432612c4ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:40.425020 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea0040-b2df-4db5-a697-2432612c4ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:40.425288 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea0040-b2df-4db5-a697-2432612c4ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:40.425586 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea0040-b2df-4db5-a697-2432612c4ef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:40.426292 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ea0040-b2df-4db5-a697-2432612c4ef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:40.427942 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ea0040-b2df-4db5-a697-2432612c4ef7 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:40.438972 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1227/2456] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:05:40.443351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fcc39c-f863-46b9-a97c-33d5e30ab7f3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:40.453788 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fcc39c-f863-46b9-a97c-33d5e30ab7f3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:40.460449 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6ce67-6d53-4ac8-9b6c-67c265997848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:40.461357 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6ce67-6d53-4ac8-9b6c-67c265997848 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:40.461703 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6ce67-6d53-4ac8-9b6c-67c265997848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:40.462102 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6ce67-6d53-4ac8-9b6c-67c265997848 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:40.463113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6ce67-6d53-4ac8-9b6c-67c265997848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:40.465492 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6ce67-6d53-4ac8-9b6c-67c265997848 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:40.479848 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1230/2457] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:05:40.485136 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea0040-b2df-4db5-a697-2432612c4ef7 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:40.501215 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea0040-b2df-4db5-a697-2432612c4ef7 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:40.508675 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e595c9-329f-42e8-b7ca-2812c6f41602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:40.509320 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e595c9-329f-42e8-b7ca-2812c6f41602 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:40.509606 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e595c9-329f-42e8-b7ca-2812c6f41602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:40.509878 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e595c9-329f-42e8-b7ca-2812c6f41602 admin admin] PATH_INFO: `/v3/projects/666fd57821744dedb9c799ad75f36b92/users/47ce52d64a1c4d41a31de6f76936ec98/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:40.527499 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e595c9-329f-42e8-b7ca-2812c6f41602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=666fd57821744dedb9c799ad75f36b92, user_id=47ce52d64a1c4d41a31de6f76936ec98, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:40.530245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e595c9-329f-42e8-b7ca-2812c6f41602 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:40.546568 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1228/2458] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:05:40 2022] PUT /identity/v3/projects/666fd57821744dedb9c799ad75f36b92/users/47ce52d64a1c4d41a31de6f76936ec98/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:05:40.559494 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9f540-1d4a-4fa0-8e08-f2cab2f10572 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:40.559939 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9f540-1d4a-4fa0-8e08-f2cab2f10572 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:40.560279 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9f540-1d4a-4fa0-8e08-f2cab2f10572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:40.601243 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b5f9f540-1d4a-4fa0-8e08-f2cab2f10572 None None] MFA Rules not processed for user `47ce52d64a1c4d41a31de6f76936ec98`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:05:40.602677 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b5f9f540-1d4a-4fa0-8e08-f2cab2f10572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:40.681496 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1231/2459] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:05:40 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:05:40.686434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e595c9-329f-42e8-b7ca-2812c6f41602 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:40.754013 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e595c9-329f-42e8-b7ca-2812c6f41602 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:40.758872 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d8a4a-c95a-4eb9-9a95-e9a559cfb00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:40.759497 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d8a4a-c95a-4eb9-9a95-e9a559cfb00b admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:40.759809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d8a4a-c95a-4eb9-9a95-e9a559cfb00b admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:40.760081 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d8a4a-c95a-4eb9-9a95-e9a559cfb00b admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:40.760754 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d8a4a-c95a-4eb9-9a95-e9a559cfb00b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:40.762699 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d8a4a-c95a-4eb9-9a95-e9a559cfb00b admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:40.793339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1229/2460] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:05:40 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:05:40.797296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9f540-1d4a-4fa0-8e08-f2cab2f10572 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:40.808557 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9f540-1d4a-4fa0-8e08-f2cab2f10572 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:40.866796 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2937a40a-c507-4eb6-a570-88deb2234933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:40.867431 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937a40a-c507-4eb6-a570-88deb2234933 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:40.867741 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937a40a-c507-4eb6-a570-88deb2234933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:40.868014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937a40a-c507-4eb6-a570-88deb2234933 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:40.868863 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2937a40a-c507-4eb6-a570-88deb2234933 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:40.870786 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2937a40a-c507-4eb6-a570-88deb2234933 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:40.880247 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:40.896892 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1232/2461] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:05:40 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:05:40.901042 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d8a4a-c95a-4eb9-9a95-e9a559cfb00b admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:40.911571 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d8a4a-c95a-4eb9-9a95-e9a559cfb00b admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:40.916355 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f4873a-70ca-43b5-956c-40f95d0b359c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:40.916966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4873a-70ca-43b5-956c-40f95d0b359c admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:40.917243 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4873a-70ca-43b5-956c-40f95d0b359c admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:40.917513 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4873a-70ca-43b5-956c-40f95d0b359c admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:40.918212 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f4873a-70ca-43b5-956c-40f95d0b359c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:40.919861 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f4873a-70ca-43b5-956c-40f95d0b359c admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:40.930815 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1230/2462] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:05:40.937440 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2937a40a-c507-4eb6-a570-88deb2234933 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:40.949506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2937a40a-c507-4eb6-a570-88deb2234933 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:40.955290 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4ad07-c333-465f-b43a-8e2780f15b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:40.956003 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4ad07-c333-465f-b43a-8e2780f15b8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:40.956308 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4ad07-c333-465f-b43a-8e2780f15b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:40.956579 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4ad07-c333-465f-b43a-8e2780f15b8f admin admin] PATH_INFO: `/v3/projects/90dd638cd9ba40a1ae47a7930a0a8f6e/users/23fb89df6bf249c9a28f8b9eb4c6e6fc/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:40.977250 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4ad07-c333-465f-b43a-8e2780f15b8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=90dd638cd9ba40a1ae47a7930a0a8f6e, user_id=23fb89df6bf249c9a28f8b9eb4c6e6fc, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:40.979969 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4ad07-c333-465f-b43a-8e2780f15b8f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:40.995685 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1233/2463] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:05:40 2022] PUT /identity/v3/projects/90dd638cd9ba40a1ae47a7930a0a8f6e/users/23fb89df6bf249c9a28f8b9eb4c6e6fc/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:05:40.999857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f4873a-70ca-43b5-956c-40f95d0b359c admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:41.010606 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f4873a-70ca-43b5-956c-40f95d0b359c admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:41.078401 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656dea47-bbd6-4c46-9b7e-943235259ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:41.079024 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656dea47-bbd6-4c46-9b7e-943235259ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:41.079305 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656dea47-bbd6-4c46-9b7e-943235259ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:41.079592 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656dea47-bbd6-4c46-9b7e-943235259ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:41.080308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656dea47-bbd6-4c46-9b7e-943235259ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:41.081906 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656dea47-bbd6-4c46-9b7e-943235259ce0 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:41.092095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1231/2464] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:05:41.098215 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4ad07-c333-465f-b43a-8e2780f15b8f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:41.109634 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4ad07-c333-465f-b43a-8e2780f15b8f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:41.116256 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e7a90e-628d-49b6-b907-4c947bb4f66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:41.116935 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7a90e-628d-49b6-b907-4c947bb4f66f admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:41.117220 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7a90e-628d-49b6-b907-4c947bb4f66f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:41.117494 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7a90e-628d-49b6-b907-4c947bb4f66f admin admin] PATH_INFO: `/v3/projects/90dd638cd9ba40a1ae47a7930a0a8f6e/users/23fb89df6bf249c9a28f8b9eb4c6e6fc/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:41.120482 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e7a90e-628d-49b6-b907-4c947bb4f66f admin admin] RBAC: Authorizing `identity:create_grant(project_id=90dd638cd9ba40a1ae47a7930a0a8f6e, user_id=23fb89df6bf249c9a28f8b9eb4c6e6fc, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:41.124282 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e7a90e-628d-49b6-b907-4c947bb4f66f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:41.137785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1234/2465] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:05:41 2022] PUT /identity/v3/projects/90dd638cd9ba40a1ae47a7930a0a8f6e/users/23fb89df6bf249c9a28f8b9eb4c6e6fc/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:05:41.149384 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a95d98-5dca-4216-a387-2c5cccf71203 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:41.149735 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a95d98-5dca-4216-a387-2c5cccf71203 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:41.150145 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a95d98-5dca-4216-a387-2c5cccf71203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:41.182608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-f7a95d98-5dca-4216-a387-2c5cccf71203 None None] MFA Rules not processed for user `23fb89df6bf249c9a28f8b9eb4c6e6fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:05:41.183909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-f7a95d98-5dca-4216-a387-2c5cccf71203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:41.199217 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:41.275536 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1232/2466] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:05:41 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:05:41.290514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e7a90e-628d-49b6-b907-4c947bb4f66f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:41.292450 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-27e7a90e-628d-49b6-b907-4c947bb4f66f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:41.355912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e7a90e-628d-49b6-b907-4c947bb4f66f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:41.360248 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1298e3a-f4dd-432c-8940-8ed41df09076 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:41.360887 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1298e3a-f4dd-432c-8940-8ed41df09076 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:41.361167 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1298e3a-f4dd-432c-8940-8ed41df09076 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:41.361440 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1298e3a-f4dd-432c-8940-8ed41df09076 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:41.371099 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1298e3a-f4dd-432c-8940-8ed41df09076 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:41.372575 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1298e3a-f4dd-432c-8940-8ed41df09076 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:41.382396 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1235/2467] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:05:41 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:05:41.837525 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a95d98-5dca-4216-a387-2c5cccf71203 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:41.909672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a95d98-5dca-4216-a387-2c5cccf71203 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:41.916191 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d440a1-4f17-4968-abeb-334e6ee29d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:41.916856 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d440a1-4f17-4968-abeb-334e6ee29d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:41.917131 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d440a1-4f17-4968-abeb-334e6ee29d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:41.917409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d440a1-4f17-4968-abeb-334e6ee29d14 admin admin] PATH_INFO: `/v3/users/47ce52d64a1c4d41a31de6f76936ec98` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:41.918745 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d440a1-4f17-4968-abeb-334e6ee29d14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47ce52d64a1c4d41a31de6f76936ec98)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:41.920754 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d440a1-4f17-4968-abeb-334e6ee29d14 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:41.974747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c3d440a1-4f17-4968-abeb-334e6ee29d14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47ce52d64a1c4d41a31de6f76936ec98'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:41.984145 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c3d440a1-4f17-4968-abeb-334e6ee29d14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47ce52d64a1c4d41a31de6f76936ec98'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:41.993795 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c3d440a1-4f17-4968-abeb-334e6ee29d14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47ce52d64a1c4d41a31de6f76936ec98'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:41.994175 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c3d440a1-4f17-4968-abeb-334e6ee29d14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47ce52d64a1c4d41a31de6f76936ec98'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:41.994889 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c3d440a1-4f17-4968-abeb-334e6ee29d14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47ce52d64a1c4d41a31de6f76936ec98'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:42.014833 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1233/2468] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:05:41 2022] DELETE /identity/v3/users/47ce52d64a1c4d41a31de6f76936ec98 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:05:42.019430 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f929fbc-0803-49e6-876a-50734c318d6e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:42.019947 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f929fbc-0803-49e6-876a-50734c318d6e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:42.020160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f929fbc-0803-49e6-876a-50734c318d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:42.040155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6f929fbc-0803-49e6-876a-50734c318d6e None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:05:42.041484 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6f929fbc-0803-49e6-876a-50734c318d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:42.104366 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1236/2469] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:05:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:05:42.116202 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d440a1-4f17-4968-abeb-334e6ee29d14 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:42.117748 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c3d440a1-4f17-4968-abeb-334e6ee29d14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:42.154534 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:42.185307 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d440a1-4f17-4968-abeb-334e6ee29d14 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:42.190383 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b9bae-6bff-4536-84cd-99068745ef95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:42.191034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b9bae-6bff-4536-84cd-99068745ef95 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:42.191318 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b9bae-6bff-4536-84cd-99068745ef95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:42.191645 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b9bae-6bff-4536-84cd-99068745ef95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:42.202440 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b9bae-6bff-4536-84cd-99068745ef95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:42.203994 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b9bae-6bff-4536-84cd-99068745ef95 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:42.213850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1234/2470] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:05:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:05:42.501576 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f929fbc-0803-49e6-876a-50734c318d6e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:42.503129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6f929fbc-0803-49e6-876a-50734c318d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:42.518496 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f929fbc-0803-49e6-876a-50734c318d6e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:42.523368 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075eee75-2af3-4a76-ae89-9738a45640a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:42.524062 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075eee75-2af3-4a76-ae89-9738a45640a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:42.524342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075eee75-2af3-4a76-ae89-9738a45640a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:42.524616 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075eee75-2af3-4a76-ae89-9738a45640a0 admin admin] PATH_INFO: `/v3/projects/666fd57821744dedb9c799ad75f36b92` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:42.525913 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075eee75-2af3-4a76-ae89-9738a45640a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=666fd57821744dedb9c799ad75f36b92)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:42.527990 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075eee75-2af3-4a76-ae89-9738a45640a0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:42.587648 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-075eee75-2af3-4a76-ae89-9738a45640a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '666fd57821744dedb9c799ad75f36b92', 'request_id': 'req-075eee75-2af3-4a76-ae89-9738a45640a0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:42.589540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=4,keystone:SELECT=92,keystone:DELETE=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:42.595989 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-075eee75-2af3-4a76-ae89-9738a45640a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '666fd57821744dedb9c799ad75f36b92', 'request_id': 'req-075eee75-2af3-4a76-ae89-9738a45640a0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:42.607076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-075eee75-2af3-4a76-ae89-9738a45640a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '666fd57821744dedb9c799ad75f36b92', 'request_id': 'req-075eee75-2af3-4a76-ae89-9738a45640a0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:42.623011 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-075eee75-2af3-4a76-ae89-9738a45640a0 admin admin] The token cache is being invalidate because project 666fd57821744dedb9c799ad75f36b92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:05:42.623416 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-075eee75-2af3-4a76-ae89-9738a45640a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:42.624427 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-075eee75-2af3-4a76-ae89-9738a45640a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:42.626187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1237/2471] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:05:42 2022] DELETE /identity/v3/projects/666fd57821744dedb9c799ad75f36b92 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:05:42.631131 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b9bae-6bff-4536-84cd-99068745ef95 service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:42.641160 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b9bae-6bff-4536-84cd-99068745ef95 service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:42.701611 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dafe4e6-587f-49fd-95d6-4372f12ce506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:42.702257 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafe4e6-587f-49fd-95d6-4372f12ce506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:42.702544 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafe4e6-587f-49fd-95d6-4372f12ce506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:42.702827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafe4e6-587f-49fd-95d6-4372f12ce506 admin admin] PATH_INFO: `/v3/users/23fb89df6bf249c9a28f8b9eb4c6e6fc` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:42.704042 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dafe4e6-587f-49fd-95d6-4372f12ce506 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23fb89df6bf249c9a28f8b9eb4c6e6fc)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:42.706017 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dafe4e6-587f-49fd-95d6-4372f12ce506 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:42.758911 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9dafe4e6-587f-49fd-95d6-4372f12ce506 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23fb89df6bf249c9a28f8b9eb4c6e6fc'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:42.766294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9dafe4e6-587f-49fd-95d6-4372f12ce506 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23fb89df6bf249c9a28f8b9eb4c6e6fc'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:42.777401 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9dafe4e6-587f-49fd-95d6-4372f12ce506 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23fb89df6bf249c9a28f8b9eb4c6e6fc'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:42.778172 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9dafe4e6-587f-49fd-95d6-4372f12ce506 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23fb89df6bf249c9a28f8b9eb4c6e6fc'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:42.778759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9dafe4e6-587f-49fd-95d6-4372f12ce506 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23fb89df6bf249c9a28f8b9eb4c6e6fc'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:42.796350 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1235/2472] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:05:42 2022] DELETE /identity/v3/users/23fb89df6bf249c9a28f8b9eb4c6e6fc => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:05:43.123141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075eee75-2af3-4a76-ae89-9738a45640a0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:43.124870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-075eee75-2af3-4a76-ae89-9738a45640a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:43.145335 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075eee75-2af3-4a76-ae89-9738a45640a0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:43.211691 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a53b06-c0cb-44db-afc0-58e644a6dc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:43.212361 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a53b06-c0cb-44db-afc0-58e644a6dc3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:43.212643 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a53b06-c0cb-44db-afc0-58e644a6dc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:43.212914 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a53b06-c0cb-44db-afc0-58e644a6dc3d admin admin] PATH_INFO: `/v3/projects/90dd638cd9ba40a1ae47a7930a0a8f6e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:43.214133 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a53b06-c0cb-44db-afc0-58e644a6dc3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=90dd638cd9ba40a1ae47a7930a0a8f6e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:43.216075 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a53b06-c0cb-44db-afc0-58e644a6dc3d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:43.266653 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b7a53b06-c0cb-44db-afc0-58e644a6dc3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90dd638cd9ba40a1ae47a7930a0a8f6e', 'request_id': 'req-b7a53b06-c0cb-44db-afc0-58e644a6dc3d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:43.273627 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b7a53b06-c0cb-44db-afc0-58e644a6dc3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90dd638cd9ba40a1ae47a7930a0a8f6e', 'request_id': 'req-b7a53b06-c0cb-44db-afc0-58e644a6dc3d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:43.284364 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b7a53b06-c0cb-44db-afc0-58e644a6dc3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90dd638cd9ba40a1ae47a7930a0a8f6e', 'request_id': 'req-b7a53b06-c0cb-44db-afc0-58e644a6dc3d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:43.298855 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b7a53b06-c0cb-44db-afc0-58e644a6dc3d admin admin] The token cache is being invalidate because project 90dd638cd9ba40a1ae47a7930a0a8f6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:05:43.299364 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b7a53b06-c0cb-44db-afc0-58e644a6dc3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:43.300327 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b7a53b06-c0cb-44db-afc0-58e644a6dc3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:43.302222 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1238/2473] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:05:43 2022] DELETE /identity/v3/projects/90dd638cd9ba40a1ae47a7930a0a8f6e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:05:43.308177 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dc000-88a8-4ae1-a30f-7c5062778b0d None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:43.308902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dc000-88a8-4ae1-a30f-7c5062778b0d None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:43.308902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dc000-88a8-4ae1-a30f-7c5062778b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:43.329355 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-719dc000-88a8-4ae1-a30f-7c5062778b0d None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:05:43.330602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-719dc000-88a8-4ae1-a30f-7c5062778b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:43.385401 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:43.400894 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1236/2474] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:05:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:05:43.409147 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de11416-5fd3-48ec-bd9d-eeb65b3951ec None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:43.409426 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de11416-5fd3-48ec-bd9d-eeb65b3951ec None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:43.409701 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de11416-5fd3-48ec-bd9d-eeb65b3951ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:43.439785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9de11416-5fd3-48ec-bd9d-eeb65b3951ec None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:05:43.441598 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9de11416-5fd3-48ec-bd9d-eeb65b3951ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:43.451587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1239/2475] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:05:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:05:43.456072 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719dc000-88a8-4ae1-a30f-7c5062778b0d None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:43.466662 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719dc000-88a8-4ae1-a30f-7c5062778b0d None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:43.472695 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71dab3-fd93-4de7-9d8c-38e16e3bc42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:43.473318 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71dab3-fd93-4de7-9d8c-38e16e3bc42e admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:43.473607 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71dab3-fd93-4de7-9d8c-38e16e3bc42e admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:43.473888 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71dab3-fd93-4de7-9d8c-38e16e3bc42e admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:43.474603 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c71dab3-fd93-4de7-9d8c-38e16e3bc42e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:05:43.474906 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c71dab3-fd93-4de7-9d8c-38e16e3bc42e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:43.476575 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c71dab3-fd93-4de7-9d8c-38e16e3bc42e admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:43.489005 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1237/2476] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:05:43.493523 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de11416-5fd3-48ec-bd9d-eeb65b3951ec None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:43.502962 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de11416-5fd3-48ec-bd9d-eeb65b3951ec None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:43.509734 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce9ba8c-60b9-41dc-9ea0-05e739ae6c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:43.510334 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce9ba8c-60b9-41dc-9ea0-05e739ae6c6a admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:43.510625 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce9ba8c-60b9-41dc-9ea0-05e739ae6c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:43.510891 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce9ba8c-60b9-41dc-9ea0-05e739ae6c6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:43.511576 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce9ba8c-60b9-41dc-9ea0-05e739ae6c6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:43.513520 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce9ba8c-60b9-41dc-9ea0-05e739ae6c6a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:43.545209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1240/2477] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:05:43 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:05:43.549682 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71dab3-fd93-4de7-9d8c-38e16e3bc42e admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:43.562939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71dab3-fd93-4de7-9d8c-38e16e3bc42e admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:43.624690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c9b68-f220-44c8-b8aa-8ab316bf5ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:43.625321 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c9b68-f220-44c8-b8aa-8ab316bf5ec6 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:43.625600 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c9b68-f220-44c8-b8aa-8ab316bf5ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:43.625871 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c9b68-f220-44c8-b8aa-8ab316bf5ec6 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:43.626550 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c9b68-f220-44c8-b8aa-8ab316bf5ec6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:43.628519 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c9b68-f220-44c8-b8aa-8ab316bf5ec6 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:43.659525 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1238/2478] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:05:43 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:05:43.665691 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce9ba8c-60b9-41dc-9ea0-05e739ae6c6a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:43.680151 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce9ba8c-60b9-41dc-9ea0-05e739ae6c6a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:43.685429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409917a1-e6f8-41a2-8fb7-60db7f0ac3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:43.686046 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409917a1-e6f8-41a2-8fb7-60db7f0ac3de admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:43.686328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409917a1-e6f8-41a2-8fb7-60db7f0ac3de admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:43.686601 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409917a1-e6f8-41a2-8fb7-60db7f0ac3de admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:43.687319 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409917a1-e6f8-41a2-8fb7-60db7f0ac3de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:43.689294 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409917a1-e6f8-41a2-8fb7-60db7f0ac3de admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:43.701752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1241/2479] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:05:43.706223 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c9b68-f220-44c8-b8aa-8ab316bf5ec6 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:43.718011 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c9b68-f220-44c8-b8aa-8ab316bf5ec6 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:43.724600 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6fa14-7e89-494a-9b62-9e40c6d93ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:43.725244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6fa14-7e89-494a-9b62-9e40c6d93ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:43.725528 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6fa14-7e89-494a-9b62-9e40c6d93ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:43.725792 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6fa14-7e89-494a-9b62-9e40c6d93ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:43.726561 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a6fa14-7e89-494a-9b62-9e40c6d93ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:43.728211 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a6fa14-7e89-494a-9b62-9e40c6d93ca3 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:43.739680 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1239/2480] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:05:43.744960 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409917a1-e6f8-41a2-8fb7-60db7f0ac3de admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:43.761944 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409917a1-e6f8-41a2-8fb7-60db7f0ac3de admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:43.770469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73676d35-a558-4850-9d42-84c124529058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:43.771142 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73676d35-a558-4850-9d42-84c124529058 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:43.771438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73676d35-a558-4850-9d42-84c124529058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:43.771741 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73676d35-a558-4850-9d42-84c124529058 admin admin] PATH_INFO: `/v3/projects/b221cad9eadc4b57b247caeec686c3ac/users/ea9e349646a14709add158a360e02cbf/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:43.788353 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73676d35-a558-4850-9d42-84c124529058 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b221cad9eadc4b57b247caeec686c3ac, user_id=ea9e349646a14709add158a360e02cbf, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:43.791186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73676d35-a558-4850-9d42-84c124529058 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:43.807081 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1242/2481] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:05:43 2022] PUT /identity/v3/projects/b221cad9eadc4b57b247caeec686c3ac/users/ea9e349646a14709add158a360e02cbf/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:05:43.819480 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b009590-e28a-4686-92e9-c2fa1ab19e93 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:43.819795 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b009590-e28a-4686-92e9-c2fa1ab19e93 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:43.820072 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b009590-e28a-4686-92e9-c2fa1ab19e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:43.847476 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-0b009590-e28a-4686-92e9-c2fa1ab19e93 None None] MFA Rules not processed for user `ea9e349646a14709add158a360e02cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:05:43.848717 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-0b009590-e28a-4686-92e9-c2fa1ab19e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:43.915061 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1240/2482] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:05:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:05:43.920069 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73676d35-a558-4850-9d42-84c124529058 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:43.929967 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:43.989478 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73676d35-a558-4850-9d42-84c124529058 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:43.994833 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a399d0-74c7-4ebb-8803-1f55f8acca3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:43.995432 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a399d0-74c7-4ebb-8803-1f55f8acca3b admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:43.995732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a399d0-74c7-4ebb-8803-1f55f8acca3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:43.996014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a399d0-74c7-4ebb-8803-1f55f8acca3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:43.996686 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a399d0-74c7-4ebb-8803-1f55f8acca3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:43.998613 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a399d0-74c7-4ebb-8803-1f55f8acca3b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:44.032712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1243/2483] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:05:43 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:05:44.036927 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b009590-e28a-4686-92e9-c2fa1ab19e93 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:44.048864 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b009590-e28a-4686-92e9-c2fa1ab19e93 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:44.097821 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:44.113821 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec4bc1-3d97-4678-8593-ed87c728599d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:44.114453 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec4bc1-3d97-4678-8593-ed87c728599d admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:44.114747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec4bc1-3d97-4678-8593-ed87c728599d admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:44.115018 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec4bc1-3d97-4678-8593-ed87c728599d admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:44.115726 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dec4bc1-3d97-4678-8593-ed87c728599d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:44.117695 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dec4bc1-3d97-4678-8593-ed87c728599d admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:44.141299 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1241/2484] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:05:44 2022] POST /identity/v3/users => generated 300 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:05:44.146361 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a399d0-74c7-4ebb-8803-1f55f8acca3b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:44.161868 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a399d0-74c7-4ebb-8803-1f55f8acca3b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:44.167102 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2719dd3-758e-4903-b421-eeb3284626f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:44.167758 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2719dd3-758e-4903-b421-eeb3284626f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:44.168047 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2719dd3-758e-4903-b421-eeb3284626f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:44.168322 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2719dd3-758e-4903-b421-eeb3284626f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:44.169062 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2719dd3-758e-4903-b421-eeb3284626f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:44.170676 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2719dd3-758e-4903-b421-eeb3284626f5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:44.182460 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1244/2485] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:05:44.189289 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec4bc1-3d97-4678-8593-ed87c728599d admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:44.201983 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec4bc1-3d97-4678-8593-ed87c728599d admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:44.208615 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e57f56-1ec9-45af-8c42-be510c77e2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:44.209278 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e57f56-1ec9-45af-8c42-be510c77e2b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:44.209560 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e57f56-1ec9-45af-8c42-be510c77e2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:44.209828 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e57f56-1ec9-45af-8c42-be510c77e2b5 admin admin] PATH_INFO: `/v3/projects/1aaeecb5ebdc4d6889235bb321e0c34a/users/105d8c2497114700902db1a89062c897/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:44.228043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e57f56-1ec9-45af-8c42-be510c77e2b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aaeecb5ebdc4d6889235bb321e0c34a, user_id=105d8c2497114700902db1a89062c897, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:44.232106 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e57f56-1ec9-45af-8c42-be510c77e2b5 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:44.253348 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1242/2486] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:05:44 2022] PUT /identity/v3/projects/1aaeecb5ebdc4d6889235bb321e0c34a/users/105d8c2497114700902db1a89062c897/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:05:44.258801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2719dd3-758e-4903-b421-eeb3284626f5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:44.269392 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2719dd3-758e-4903-b421-eeb3284626f5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:44.329413 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c4506-7fee-4930-923b-d849889f7da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:44.330056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c4506-7fee-4930-923b-d849889f7da8 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:44.330343 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c4506-7fee-4930-923b-d849889f7da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:44.330621 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c4506-7fee-4930-923b-d849889f7da8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:44.331401 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c4506-7fee-4930-923b-d849889f7da8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:44.333140 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c4506-7fee-4930-923b-d849889f7da8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:44.346067 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1245/2487] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:05:44.348760 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e57f56-1ec9-45af-8c42-be510c77e2b5 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:44.359481 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e57f56-1ec9-45af-8c42-be510c77e2b5 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:44.365346 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa7c8e-9b21-4178-9681-1ce4019f5846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:44.366013 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa7c8e-9b21-4178-9681-1ce4019f5846 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:44.366276 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa7c8e-9b21-4178-9681-1ce4019f5846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:44.366548 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa7c8e-9b21-4178-9681-1ce4019f5846 admin admin] PATH_INFO: `/v3/projects/1aaeecb5ebdc4d6889235bb321e0c34a/users/105d8c2497114700902db1a89062c897/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:44.368587 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa7c8e-9b21-4178-9681-1ce4019f5846 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aaeecb5ebdc4d6889235bb321e0c34a, user_id=105d8c2497114700902db1a89062c897, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:44.371244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa7c8e-9b21-4178-9681-1ce4019f5846 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:44.382000 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1243/2488] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:05:44 2022] PUT /identity/v3/projects/1aaeecb5ebdc4d6889235bb321e0c34a/users/105d8c2497114700902db1a89062c897/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:05:44.392673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739b98e-b39b-48b2-b849-4ae60fd2067b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:44.392965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739b98e-b39b-48b2-b849-4ae60fd2067b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:44.393249 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739b98e-b39b-48b2-b849-4ae60fd2067b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:44.423995 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e739b98e-b39b-48b2-b849-4ae60fd2067b None None] MFA Rules not processed for user `105d8c2497114700902db1a89062c897`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:05:44.425171 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e739b98e-b39b-48b2-b849-4ae60fd2067b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:44.497254 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1246/2489] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:05:44 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:05:44.509505 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa7c8e-9b21-4178-9681-1ce4019f5846 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:44.512731 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-d6fa7c8e-9b21-4178-9681-1ce4019f5846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:44.577578 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa7c8e-9b21-4178-9681-1ce4019f5846 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:44.582231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1eaab-d54d-4e98-abae-57a74cc51183 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:44.582851 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1eaab-d54d-4e98-abae-57a74cc51183 service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:44.583136 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1eaab-d54d-4e98-abae-57a74cc51183 service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:44.583406 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1eaab-d54d-4e98-abae-57a74cc51183 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:44.593686 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1eaab-d54d-4e98-abae-57a74cc51183 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:44.595149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1eaab-d54d-4e98-abae-57a74cc51183 service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:44.606146 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1244/2490] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:05:44 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:05:44.640775 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739b98e-b39b-48b2-b849-4ae60fd2067b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:44.645717 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:44.708463 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739b98e-b39b-48b2-b849-4ae60fd2067b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:44.713466 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1223ef4-3333-44df-8070-73e517af1cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:44.714123 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1223ef4-3333-44df-8070-73e517af1cb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:44.714412 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1223ef4-3333-44df-8070-73e517af1cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:44.714695 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1223ef4-3333-44df-8070-73e517af1cb0 admin admin] PATH_INFO: `/v3/users/ea9e349646a14709add158a360e02cbf` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:44.716040 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1223ef4-3333-44df-8070-73e517af1cb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea9e349646a14709add158a360e02cbf)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:44.717999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1223ef4-3333-44df-8070-73e517af1cb0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:44.771214 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e1223ef4-3333-44df-8070-73e517af1cb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea9e349646a14709add158a360e02cbf'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:44.778500 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e1223ef4-3333-44df-8070-73e517af1cb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea9e349646a14709add158a360e02cbf'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:44.779224 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e1223ef4-3333-44df-8070-73e517af1cb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea9e349646a14709add158a360e02cbf'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:44.788288 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e1223ef4-3333-44df-8070-73e517af1cb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea9e349646a14709add158a360e02cbf'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:44.789023 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e1223ef4-3333-44df-8070-73e517af1cb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea9e349646a14709add158a360e02cbf'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:44.807465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1247/2491] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:05:44 2022] DELETE /identity/v3/users/ea9e349646a14709add158a360e02cbf => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:05:44.812795 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634634e-790f-4963-b793-15afe2e72d28 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:44.812795 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634634e-790f-4963-b793-15afe2e72d28 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:44.812795 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634634e-790f-4963-b793-15afe2e72d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:44.834420 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-a634634e-790f-4963-b793-15afe2e72d28 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:05:44.835758 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a634634e-790f-4963-b793-15afe2e72d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:44.896452 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1245/2492] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:05:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:05:44.906394 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1223ef4-3333-44df-8070-73e517af1cb0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:44.908193 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e1223ef4-3333-44df-8070-73e517af1cb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:45.111275 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1223ef4-3333-44df-8070-73e517af1cb0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:45.115529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012f127-8978-4551-ab5d-1249c3926275 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:45.116178 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012f127-8978-4551-ab5d-1249c3926275 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:45.116458 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012f127-8978-4551-ab5d-1249c3926275 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:45.116732 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012f127-8978-4551-ab5d-1249c3926275 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:45.129183 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1012f127-8978-4551-ab5d-1249c3926275 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:45.131279 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1012f127-8978-4551-ab5d-1249c3926275 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:45.146033 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1248/2493] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:05:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:05:45.435980 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634634e-790f-4963-b793-15afe2e72d28 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:45.437539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a634634e-790f-4963-b793-15afe2e72d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:45.450750 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634634e-790f-4963-b793-15afe2e72d28 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:45.456061 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c356df9-240d-49f6-92db-d5c7c36d4ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:45.456728 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c356df9-240d-49f6-92db-d5c7c36d4ac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:45.457012 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c356df9-240d-49f6-92db-d5c7c36d4ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:45.457284 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c356df9-240d-49f6-92db-d5c7c36d4ac8 admin admin] PATH_INFO: `/v3/projects/b221cad9eadc4b57b247caeec686c3ac` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:45.458602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c356df9-240d-49f6-92db-d5c7c36d4ac8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b221cad9eadc4b57b247caeec686c3ac)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:45.460598 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c356df9-240d-49f6-92db-d5c7c36d4ac8 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:45.510228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:45.518514 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9c356df9-240d-49f6-92db-d5c7c36d4ac8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b221cad9eadc4b57b247caeec686c3ac', 'request_id': 'req-9c356df9-240d-49f6-92db-d5c7c36d4ac8'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:45.526410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9c356df9-240d-49f6-92db-d5c7c36d4ac8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b221cad9eadc4b57b247caeec686c3ac', 'request_id': 'req-9c356df9-240d-49f6-92db-d5c7c36d4ac8'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:45.532864 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9c356df9-240d-49f6-92db-d5c7c36d4ac8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b221cad9eadc4b57b247caeec686c3ac', 'request_id': 'req-9c356df9-240d-49f6-92db-d5c7c36d4ac8'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:45.544933 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9c356df9-240d-49f6-92db-d5c7c36d4ac8 admin admin] The token cache is being invalidate because project b221cad9eadc4b57b247caeec686c3ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:05:45.545228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9c356df9-240d-49f6-92db-d5c7c36d4ac8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:45.545936 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9c356df9-240d-49f6-92db-d5c7c36d4ac8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:45.550488 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1246/2494] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:05:45 2022] DELETE /identity/v3/projects/b221cad9eadc4b57b247caeec686c3ac => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:05:45.551961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012f127-8978-4551-ab5d-1249c3926275 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:45.562331 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012f127-8978-4551-ab5d-1249c3926275 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:45.582404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c356df9-240d-49f6-92db-d5c7c36d4ac8 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:45.584386 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-9c356df9-240d-49f6-92db-d5c7c36d4ac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:45.587226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:45.634745 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78c8bb-711c-48b7-900b-0cd08d99ef57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:45.635797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78c8bb-711c-48b7-900b-0cd08d99ef57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:45.636088 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78c8bb-711c-48b7-900b-0cd08d99ef57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:45.636525 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78c8bb-711c-48b7-900b-0cd08d99ef57 admin admin] PATH_INFO: `/v3/users/105d8c2497114700902db1a89062c897` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:45.637916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78c8bb-711c-48b7-900b-0cd08d99ef57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=105d8c2497114700902db1a89062c897)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:45.639749 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78c8bb-711c-48b7-900b-0cd08d99ef57 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:45.665923 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c356df9-240d-49f6-92db-d5c7c36d4ac8 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:45.672564 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c0454c-a1b6-4912-9856-c5ede6360fc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:45.673587 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0454c-a1b6-4912-9856-c5ede6360fc1 service nova] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:45.673587 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0454c-a1b6-4912-9856-c5ede6360fc1 service nova] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:45.673895 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0454c-a1b6-4912-9856-c5ede6360fc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:45.675842 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-d4c0454c-a1b6-4912-9856-c5ede6360fc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:45.692338 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8d78c8bb-711c-48b7-900b-0cd08d99ef57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '105d8c2497114700902db1a89062c897'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:45.700436 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8d78c8bb-711c-48b7-900b-0cd08d99ef57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '105d8c2497114700902db1a89062c897'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:45.701172 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8d78c8bb-711c-48b7-900b-0cd08d99ef57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '105d8c2497114700902db1a89062c897'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:45.710903 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8d78c8bb-711c-48b7-900b-0cd08d99ef57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '105d8c2497114700902db1a89062c897'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:45.711638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8d78c8bb-711c-48b7-900b-0cd08d99ef57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '105d8c2497114700902db1a89062c897'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:45.731184 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1249/2495] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:05:45 2022] DELETE /identity/v3/users/105d8c2497114700902db1a89062c897 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:05:45.740033 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c0454c-a1b6-4912-9856-c5ede6360fc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:45.741665 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c0454c-a1b6-4912-9856-c5ede6360fc1 service nova] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:45.751781 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1247/2496] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:05:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:05:46.014478 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78c8bb-711c-48b7-900b-0cd08d99ef57 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:46.016789 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8d78c8bb-711c-48b7-900b-0cd08d99ef57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:46.093294 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78c8bb-711c-48b7-900b-0cd08d99ef57 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:46.098086 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579641e6-109f-407d-aeff-e545e915114e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:46.098830 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579641e6-109f-407d-aeff-e545e915114e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:46.099175 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579641e6-109f-407d-aeff-e545e915114e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:46.099511 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579641e6-109f-407d-aeff-e545e915114e admin admin] PATH_INFO: `/v3/projects/1aaeecb5ebdc4d6889235bb321e0c34a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:46.100839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579641e6-109f-407d-aeff-e545e915114e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1aaeecb5ebdc4d6889235bb321e0c34a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:46.103347 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579641e6-109f-407d-aeff-e545e915114e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:46.149994 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:46.153259 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-579641e6-109f-407d-aeff-e545e915114e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1aaeecb5ebdc4d6889235bb321e0c34a', 'request_id': 'req-579641e6-109f-407d-aeff-e545e915114e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:46.161487 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-579641e6-109f-407d-aeff-e545e915114e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1aaeecb5ebdc4d6889235bb321e0c34a', 'request_id': 'req-579641e6-109f-407d-aeff-e545e915114e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:46.169957 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-579641e6-109f-407d-aeff-e545e915114e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1aaeecb5ebdc4d6889235bb321e0c34a', 'request_id': 'req-579641e6-109f-407d-aeff-e545e915114e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:46.180409 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-579641e6-109f-407d-aeff-e545e915114e admin admin] The token cache is being invalidate because project 1aaeecb5ebdc4d6889235bb321e0c34a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:05:46.180699 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-579641e6-109f-407d-aeff-e545e915114e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:46.181366 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-579641e6-109f-407d-aeff-e545e915114e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:46.182978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1250/2497] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:05:46 2022] DELETE /identity/v3/projects/1aaeecb5ebdc4d6889235bb321e0c34a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:05:46.188742 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7177e3-2782-4065-b831-383bcd19183d None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:46.189026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7177e3-2782-4065-b831-383bcd19183d None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:46.189299 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7177e3-2782-4065-b831-383bcd19183d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:46.210971 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-9e7177e3-2782-4065-b831-383bcd19183d None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:05:46.212124 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-9e7177e3-2782-4065-b831-383bcd19183d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:46.275018 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1248/2498] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:05:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:05:46.283680 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16341ac2-c5b3-4771-b129-b46ef79fad8e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:46.283981 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16341ac2-c5b3-4771-b129-b46ef79fad8e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:46.284251 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16341ac2-c5b3-4771-b129-b46ef79fad8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:46.305443 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-16341ac2-c5b3-4771-b129-b46ef79fad8e None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:05:46.306662 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-16341ac2-c5b3-4771-b129-b46ef79fad8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:46.316349 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1251/2499] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:05:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:05:46.320963 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7177e3-2782-4065-b831-383bcd19183d None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:46.327316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:46.331511 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7177e3-2782-4065-b831-383bcd19183d None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:46.338184 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b0943-3ebe-42ec-a2a6-89c9d6a438c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:46.338826 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b0943-3ebe-42ec-a2a6-89c9d6a438c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:46.339365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b0943-3ebe-42ec-a2a6-89c9d6a438c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:46.339902 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b0943-3ebe-42ec-a2a6-89c9d6a438c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:46.340685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b0943-3ebe-42ec-a2a6-89c9d6a438c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:05:46.341193 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b0943-3ebe-42ec-a2a6-89c9d6a438c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:46.342875 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b0943-3ebe-42ec-a2a6-89c9d6a438c8 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:46.354923 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1249/2500] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:05:46.359594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16341ac2-c5b3-4771-b129-b46ef79fad8e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:46.368809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16341ac2-c5b3-4771-b129-b46ef79fad8e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:46.374419 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6541d25-3248-4815-8fad-85fa4badfd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:46.375040 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6541d25-3248-4815-8fad-85fa4badfd2d admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:46.375438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6541d25-3248-4815-8fad-85fa4badfd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:46.375750 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6541d25-3248-4815-8fad-85fa4badfd2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:46.376413 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6541d25-3248-4815-8fad-85fa4badfd2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:46.378308 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6541d25-3248-4815-8fad-85fa4badfd2d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:46.412796 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1252/2501] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:05:46 2022] POST /identity/v3/projects => generated 393 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:05:46.417200 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b0943-3ebe-42ec-a2a6-89c9d6a438c8 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:46.427457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b0943-3ebe-42ec-a2a6-89c9d6a438c8 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:46.483498 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45026554-a848-4d5d-b24d-5977c72fa841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:46.484198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45026554-a848-4d5d-b24d-5977c72fa841 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:46.484297 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45026554-a848-4d5d-b24d-5977c72fa841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:46.484571 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45026554-a848-4d5d-b24d-5977c72fa841 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:46.485231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45026554-a848-4d5d-b24d-5977c72fa841 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:46.487153 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45026554-a848-4d5d-b24d-5977c72fa841 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:46.510225 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1250/2502] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:05:46 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:05:46.514607 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6541d25-3248-4815-8fad-85fa4badfd2d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:46.525668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6541d25-3248-4815-8fad-85fa4badfd2d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:46.531677 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7810516-b993-4dfc-b5ba-0b59ac7cde9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:46.532429 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7810516-b993-4dfc-b5ba-0b59ac7cde9d admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:46.532752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7810516-b993-4dfc-b5ba-0b59ac7cde9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:46.532984 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7810516-b993-4dfc-b5ba-0b59ac7cde9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:46.533685 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7810516-b993-4dfc-b5ba-0b59ac7cde9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:46.535365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7810516-b993-4dfc-b5ba-0b59ac7cde9d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:46.551172 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1253/2503] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:05:46.557506 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45026554-a848-4d5d-b24d-5977c72fa841 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:46.570681 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45026554-a848-4d5d-b24d-5977c72fa841 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:46.576625 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a9283-ace9-43b6-96a1-4ed0e573e589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:46.577255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a9283-ace9-43b6-96a1-4ed0e573e589 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:46.577538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a9283-ace9-43b6-96a1-4ed0e573e589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:46.577812 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a9283-ace9-43b6-96a1-4ed0e573e589 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:46.578537 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6a9283-ace9-43b6-96a1-4ed0e573e589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:46.580447 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6a9283-ace9-43b6-96a1-4ed0e573e589 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:46.592117 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1251/2504] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:05:46.596736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7810516-b993-4dfc-b5ba-0b59ac7cde9d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:46.606673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7810516-b993-4dfc-b5ba-0b59ac7cde9d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:46.611532 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b1efd-eb2b-471c-bbd9-73d4257b3fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:46.612380 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b1efd-eb2b-471c-bbd9-73d4257b3fe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:46.612673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b1efd-eb2b-471c-bbd9-73d4257b3fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:46.613042 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b1efd-eb2b-471c-bbd9-73d4257b3fe6 admin admin] PATH_INFO: `/v3/projects/cbb1922139b9478f8b15dfffc8336b57/users/c055e613a5c64b51aaadd2ccd7c39725/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:46.631137 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b1efd-eb2b-471c-bbd9-73d4257b3fe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbb1922139b9478f8b15dfffc8336b57, user_id=c055e613a5c64b51aaadd2ccd7c39725, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:46.635103 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b1efd-eb2b-471c-bbd9-73d4257b3fe6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:46.651221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1254/2505] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:05:46 2022] PUT /identity/v3/projects/cbb1922139b9478f8b15dfffc8336b57/users/c055e613a5c64b51aaadd2ccd7c39725/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:05:46.662961 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8324c4-b582-4c82-8476-2c6b6d14da51 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:46.663332 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8324c4-b582-4c82-8476-2c6b6d14da51 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:46.663718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8324c4-b582-4c82-8476-2c6b6d14da51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:46.694884 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-8c8324c4-b582-4c82-8476-2c6b6d14da51 None None] MFA Rules not processed for user `c055e613a5c64b51aaadd2ccd7c39725`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:05:46.696225 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-8c8324c4-b582-4c82-8476-2c6b6d14da51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:46.770717 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1252/2506] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:05:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:05:46.774924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b1efd-eb2b-471c-bbd9-73d4257b3fe6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:46.840303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b1efd-eb2b-471c-bbd9-73d4257b3fe6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:46.845000 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905df87b-618c-4aa1-9241-3db224a74ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:46.845660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905df87b-618c-4aa1-9241-3db224a74ad7 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:46.845946 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905df87b-618c-4aa1-9241-3db224a74ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:46.846219 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905df87b-618c-4aa1-9241-3db224a74ad7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:46.846918 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905df87b-618c-4aa1-9241-3db224a74ad7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:46.848902 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905df87b-618c-4aa1-9241-3db224a74ad7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:46.876988 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1255/2507] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:05:46 2022] POST /identity/v3/projects => generated 393 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:05:46.881089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8324c4-b582-4c82-8476-2c6b6d14da51 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:46.892356 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8324c4-b582-4c82-8476-2c6b6d14da51 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:46.957870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4310b2-5a80-4d79-9986-2bf281b6a71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:46.958527 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4310b2-5a80-4d79-9986-2bf281b6a71a admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:46.958933 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4310b2-5a80-4d79-9986-2bf281b6a71a admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:46.959209 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4310b2-5a80-4d79-9986-2bf281b6a71a admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:46.960311 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4310b2-5a80-4d79-9986-2bf281b6a71a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:46.962369 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4310b2-5a80-4d79-9986-2bf281b6a71a admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:46.971406 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:46.988715 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1253/2508] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:05:46 2022] POST /identity/v3/users => generated 303 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:05:46.992888 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905df87b-618c-4aa1-9241-3db224a74ad7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:47.003313 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905df87b-618c-4aa1-9241-3db224a74ad7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:47.010989 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a76d9b-2689-45a9-a7ad-a277c90dda00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:47.011683 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a76d9b-2689-45a9-a7ad-a277c90dda00 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:47.011940 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a76d9b-2689-45a9-a7ad-a277c90dda00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:47.012212 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a76d9b-2689-45a9-a7ad-a277c90dda00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:47.012980 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a76d9b-2689-45a9-a7ad-a277c90dda00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:47.014600 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a76d9b-2689-45a9-a7ad-a277c90dda00 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:47.025595 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1256/2509] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:05:47.030187 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4310b2-5a80-4d79-9986-2bf281b6a71a admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:47.040460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4310b2-5a80-4d79-9986-2bf281b6a71a admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:47.043782 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c80255-75a3-4ea3-9017-d0cdfe52380c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:47.044148 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c80255-75a3-4ea3-9017-d0cdfe52380c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:47.044609 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c80255-75a3-4ea3-9017-d0cdfe52380c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:47.045668 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf428ccf-64be-489a-b0fe-7f224d6449bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:47.046379 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf428ccf-64be-489a-b0fe-7f224d6449bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:47.046633 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf428ccf-64be-489a-b0fe-7f224d6449bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:47.046912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf428ccf-64be-489a-b0fe-7f224d6449bf admin admin] PATH_INFO: `/v3/projects/3c5e926bc18b4132b592418f6170c081/users/b22208b162d34e09979df4035d6fbc2a/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:47.062873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf428ccf-64be-489a-b0fe-7f224d6449bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c5e926bc18b4132b592418f6170c081, user_id=b22208b162d34e09979df4035d6fbc2a, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:47.065076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a7c80255-75a3-4ea3-9017-d0cdfe52380c None None] MFA Rules not processed for user `3a8f3eb811ae40ad9cb64261f95a992e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:05:47.066073 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf428ccf-64be-489a-b0fe-7f224d6449bf admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:47.066347 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a7c80255-75a3-4ea3-9017-d0cdfe52380c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:47.080874 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1254/2510] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:05:47 2022] PUT /identity/v3/projects/3c5e926bc18b4132b592418f6170c081/users/b22208b162d34e09979df4035d6fbc2a/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:05:47.085136 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf428ccf-64be-489a-b0fe-7f224d6449bf admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:47.117281 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:47.136853 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1257/2511] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jul 26 05:05:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:05:47.166126 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf428ccf-64be-489a-b0fe-7f224d6449bf admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:47.171955 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09303aba-9e7e-421b-a599-a084b3634cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:47.172710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09303aba-9e7e-421b-a599-a084b3634cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:47.173052 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09303aba-9e7e-421b-a599-a084b3634cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:47.173518 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09303aba-9e7e-421b-a599-a084b3634cf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:47.174280 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09303aba-9e7e-421b-a599-a084b3634cf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:47.176098 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09303aba-9e7e-421b-a599-a084b3634cf5 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:47.187433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1255/2512] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:05:47.191929 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c80255-75a3-4ea3-9017-d0cdfe52380c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:47.194789 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09303aba-9e7e-421b-a599-a084b3634cf5 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:47.196810 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-09303aba-9e7e-421b-a599-a084b3634cf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:47.204723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c80255-75a3-4ea3-9017-d0cdfe52380c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:47.212591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f44419-f89c-4d5e-8c6f-b9dc51954908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:47.213631 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f44419-f89c-4d5e-8c6f-b9dc51954908 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:47.213919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f44419-f89c-4d5e-8c6f-b9dc51954908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:47.214196 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f44419-f89c-4d5e-8c6f-b9dc51954908 admin admin] PATH_INFO: `/v3/projects/3c5e926bc18b4132b592418f6170c081/users/b22208b162d34e09979df4035d6fbc2a/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:47.216297 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f44419-f89c-4d5e-8c6f-b9dc51954908 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c5e926bc18b4132b592418f6170c081, user_id=b22208b162d34e09979df4035d6fbc2a, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:47.219503 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f44419-f89c-4d5e-8c6f-b9dc51954908 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:47.233534 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1258/2513] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:05:47 2022] PUT /identity/v3/projects/3c5e926bc18b4132b592418f6170c081/users/b22208b162d34e09979df4035d6fbc2a/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:05:47.245708 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea14cc-b02a-4ef6-8e30-fe7118fdd91a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:47.246461 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea14cc-b02a-4ef6-8e30-fe7118fdd91a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:47.246742 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea14cc-b02a-4ef6-8e30-fe7118fdd91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:47.276271 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-10ea14cc-b02a-4ef6-8e30-fe7118fdd91a None None] MFA Rules not processed for user `b22208b162d34e09979df4035d6fbc2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:05:47.276692 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09303aba-9e7e-421b-a599-a084b3634cf5 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:47.277951 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-10ea14cc-b02a-4ef6-8e30-fe7118fdd91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:47.351381 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce77457-0a49-4e8b-9a93-4939de3a1ae0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5218319aaa5424882714bffcbeee730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:47.352167 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce77457-0a49-4e8b-9a93-4939de3a1ae0 service swift] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:47.352332 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce77457-0a49-4e8b-9a93-4939de3a1ae0 service swift] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:47.352610 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce77457-0a49-4e8b-9a93-4939de3a1ae0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:47.358309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1259/2514] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:05:47 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:05:47.360832 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:47.372669 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea14cc-b02a-4ef6-8e30-fe7118fdd91a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:47.374069 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-10ea14cc-b02a-4ef6-8e30-fe7118fdd91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:47.431995 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce77457-0a49-4e8b-9a93-4939de3a1ae0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:47.433996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce77457-0a49-4e8b-9a93-4939de3a1ae0 service swift] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:47.444996 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea14cc-b02a-4ef6-8e30-fe7118fdd91a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:47.446276 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1256/2515] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jul 26 05:05:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 05:05:47.452395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e7283-22ba-4e5c-8f67-3d5d3f4152d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:47.453170 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e7283-22ba-4e5c-8f67-3d5d3f4152d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:47.453450 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e7283-22ba-4e5c-8f67-3d5d3f4152d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:47.456011 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e7283-22ba-4e5c-8f67-3d5d3f4152d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:47.468739 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e7283-22ba-4e5c-8f67-3d5d3f4152d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:47.471103 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e7283-22ba-4e5c-8f67-3d5d3f4152d5 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:47.486683 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1260/2516] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:05:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:05:47.544982 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce77457-0a49-4e8b-9a93-4939de3a1ae0 service swift] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:47.561999 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce77457-0a49-4e8b-9a93-4939de3a1ae0 service swift] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:47.569280 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4508ef82-47f7-4486-999d-0ee8b9e14b3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:47.570286 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508ef82-47f7-4486-999d-0ee8b9e14b3b service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:47.570662 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508ef82-47f7-4486-999d-0ee8b9e14b3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:47.571030 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508ef82-47f7-4486-999d-0ee8b9e14b3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:47.640464 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4508ef82-47f7-4486-999d-0ee8b9e14b3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:47.642859 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4508ef82-47f7-4486-999d-0ee8b9e14b3b service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:47.655665 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1257/2517] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:05:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:05:51.331438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e7283-22ba-4e5c-8f67-3d5d3f4152d5 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:51.409525 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e7283-22ba-4e5c-8f67-3d5d3f4152d5 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:51.414147 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69788dcd-8fd5-4119-9406-a2f917f946f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:51.414871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69788dcd-8fd5-4119-9406-a2f917f946f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:51.415155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69788dcd-8fd5-4119-9406-a2f917f946f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:51.415430 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69788dcd-8fd5-4119-9406-a2f917f946f7 admin admin] PATH_INFO: `/v3/users/c055e613a5c64b51aaadd2ccd7c39725` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:51.416641 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69788dcd-8fd5-4119-9406-a2f917f946f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c055e613a5c64b51aaadd2ccd7c39725)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:51.418578 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69788dcd-8fd5-4119-9406-a2f917f946f7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:51.423705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:51.480292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-69788dcd-8fd5-4119-9406-a2f917f946f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c055e613a5c64b51aaadd2ccd7c39725'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:51.493166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-69788dcd-8fd5-4119-9406-a2f917f946f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c055e613a5c64b51aaadd2ccd7c39725'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:51.493928 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-69788dcd-8fd5-4119-9406-a2f917f946f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c055e613a5c64b51aaadd2ccd7c39725'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:51.505026 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-69788dcd-8fd5-4119-9406-a2f917f946f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c055e613a5c64b51aaadd2ccd7c39725'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:51.505747 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-69788dcd-8fd5-4119-9406-a2f917f946f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c055e613a5c64b51aaadd2ccd7c39725'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:51.534471 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1261/2518] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:05:51 2022] DELETE /identity/v3/users/c055e613a5c64b51aaadd2ccd7c39725 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:05:51.538850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f969286-8e97-4d3c-a6d9-58cdc356c0e1 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:51.539355 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f969286-8e97-4d3c-a6d9-58cdc356c0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:51.539469 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f969286-8e97-4d3c-a6d9-58cdc356c0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:51.560909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-5f969286-8e97-4d3c-a6d9-58cdc356c0e1 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:05:51.562151 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-5f969286-8e97-4d3c-a6d9-58cdc356c0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:51.658653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1258/2519] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:05:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:05:51.668334 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69788dcd-8fd5-4119-9406-a2f917f946f7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:51.669848 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-69788dcd-8fd5-4119-9406-a2f917f946f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:51.778631 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69788dcd-8fd5-4119-9406-a2f917f946f7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:51.782714 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a170f5-cdad-4049-9ff4-e07c891f2b04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:51.783365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a170f5-cdad-4049-9ff4-e07c891f2b04 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:51.783687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a170f5-cdad-4049-9ff4-e07c891f2b04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:51.783963 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a170f5-cdad-4049-9ff4-e07c891f2b04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:51.795581 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a170f5-cdad-4049-9ff4-e07c891f2b04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:51.797975 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a170f5-cdad-4049-9ff4-e07c891f2b04 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:51.808344 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1262/2520] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:05:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:05:52.169131 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f969286-8e97-4d3c-a6d9-58cdc356c0e1 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:52.170915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-5f969286-8e97-4d3c-a6d9-58cdc356c0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:52.186817 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f969286-8e97-4d3c-a6d9-58cdc356c0e1 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:52.192574 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31721f17-d9f9-4e05-ac45-827bef651aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:52.193853 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31721f17-d9f9-4e05-ac45-827bef651aa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:52.194467 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31721f17-d9f9-4e05-ac45-827bef651aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:52.194755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31721f17-d9f9-4e05-ac45-827bef651aa1 admin admin] PATH_INFO: `/v3/projects/cbb1922139b9478f8b15dfffc8336b57` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:52.198927 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31721f17-d9f9-4e05-ac45-827bef651aa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbb1922139b9478f8b15dfffc8336b57)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:52.201125 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31721f17-d9f9-4e05-ac45-827bef651aa1 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:52.251215 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:52.265650 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-31721f17-d9f9-4e05-ac45-827bef651aa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbb1922139b9478f8b15dfffc8336b57', 'request_id': 'req-31721f17-d9f9-4e05-ac45-827bef651aa1'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:52.277297 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-31721f17-d9f9-4e05-ac45-827bef651aa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbb1922139b9478f8b15dfffc8336b57', 'request_id': 'req-31721f17-d9f9-4e05-ac45-827bef651aa1'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:52.286107 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-31721f17-d9f9-4e05-ac45-827bef651aa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbb1922139b9478f8b15dfffc8336b57', 'request_id': 'req-31721f17-d9f9-4e05-ac45-827bef651aa1'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:52.301773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-31721f17-d9f9-4e05-ac45-827bef651aa1 admin admin] The token cache is being invalidate because project cbb1922139b9478f8b15dfffc8336b57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:05:52.302216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-31721f17-d9f9-4e05-ac45-827bef651aa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:52.302947 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-31721f17-d9f9-4e05-ac45-827bef651aa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:52.304809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1259/2521] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:05:52 2022] DELETE /identity/v3/projects/cbb1922139b9478f8b15dfffc8336b57 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:05:52.309097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a170f5-cdad-4049-9ff4-e07c891f2b04 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:52.324181 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a170f5-cdad-4049-9ff4-e07c891f2b04 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:52.418922 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a586d88-862e-4c9b-a9dd-206cb4f6c21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:52.419968 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a586d88-862e-4c9b-a9dd-206cb4f6c21c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:52.420334 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a586d88-862e-4c9b-a9dd-206cb4f6c21c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:52.420915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a586d88-862e-4c9b-a9dd-206cb4f6c21c admin admin] PATH_INFO: `/v3/users/b22208b162d34e09979df4035d6fbc2a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:52.422329 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a586d88-862e-4c9b-a9dd-206cb4f6c21c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b22208b162d34e09979df4035d6fbc2a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:52.424433 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a586d88-862e-4c9b-a9dd-206cb4f6c21c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:52.473772 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:52.500792 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4a586d88-862e-4c9b-a9dd-206cb4f6c21c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b22208b162d34e09979df4035d6fbc2a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:52.508674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4a586d88-862e-4c9b-a9dd-206cb4f6c21c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b22208b162d34e09979df4035d6fbc2a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:52.509564 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4a586d88-862e-4c9b-a9dd-206cb4f6c21c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b22208b162d34e09979df4035d6fbc2a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:52.521004 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4a586d88-862e-4c9b-a9dd-206cb4f6c21c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b22208b162d34e09979df4035d6fbc2a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:52.521613 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4a586d88-862e-4c9b-a9dd-206cb4f6c21c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b22208b162d34e09979df4035d6fbc2a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:52.544356 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1263/2522] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:05:52 2022] DELETE /identity/v3/users/b22208b162d34e09979df4035d6fbc2a => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:05:52.966285 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31721f17-d9f9-4e05-ac45-827bef651aa1 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:52.968594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-31721f17-d9f9-4e05-ac45-827bef651aa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:52.995856 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31721f17-d9f9-4e05-ac45-827bef651aa1 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:53.077376 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737d72e-4c60-4a92-b68d-395a0d838ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:53.078130 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737d72e-4c60-4a92-b68d-395a0d838ab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:53.078419 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737d72e-4c60-4a92-b68d-395a0d838ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:53.078700 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737d72e-4c60-4a92-b68d-395a0d838ab1 admin admin] PATH_INFO: `/v3/projects/3c5e926bc18b4132b592418f6170c081` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:53.080000 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2737d72e-4c60-4a92-b68d-395a0d838ab1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c5e926bc18b4132b592418f6170c081)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:53.082484 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2737d72e-4c60-4a92-b68d-395a0d838ab1 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:53.144127 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-2737d72e-4c60-4a92-b68d-395a0d838ab1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c5e926bc18b4132b592418f6170c081', 'request_id': 'req-2737d72e-4c60-4a92-b68d-395a0d838ab1'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:53.154920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-2737d72e-4c60-4a92-b68d-395a0d838ab1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c5e926bc18b4132b592418f6170c081', 'request_id': 'req-2737d72e-4c60-4a92-b68d-395a0d838ab1'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:53.163731 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-2737d72e-4c60-4a92-b68d-395a0d838ab1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c5e926bc18b4132b592418f6170c081', 'request_id': 'req-2737d72e-4c60-4a92-b68d-395a0d838ab1'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:53.181443 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-2737d72e-4c60-4a92-b68d-395a0d838ab1 admin admin] The token cache is being invalidate because project 3c5e926bc18b4132b592418f6170c081 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:05:53.181709 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-2737d72e-4c60-4a92-b68d-395a0d838ab1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:53.182398 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-2737d72e-4c60-4a92-b68d-395a0d838ab1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:53.188320 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1260/2523] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:05:52 2022] DELETE /identity/v3/projects/3c5e926bc18b4132b592418f6170c081 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:05:53.189967 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96605b3-1672-4cf8-8e78-f1fb3236997c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:53.190288 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96605b3-1672-4cf8-8e78-f1fb3236997c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:53.190593 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96605b3-1672-4cf8-8e78-f1fb3236997c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:53.216413 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a96605b3-1672-4cf8-8e78-f1fb3236997c None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:05:53.217661 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a96605b3-1672-4cf8-8e78-f1fb3236997c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:53.292371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1264/2524] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:05:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:05:53.301282 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d37b7e-802a-41c3-936f-1646fc5f128c None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:53.301485 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d37b7e-802a-41c3-936f-1646fc5f128c None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:53.301995 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d37b7e-802a-41c3-936f-1646fc5f128c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:53.324579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-02d37b7e-802a-41c3-936f-1646fc5f128c None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:05:53.325828 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-02d37b7e-802a-41c3-936f-1646fc5f128c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:53.344050 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1261/2525] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:05:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:05:53.348171 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96605b3-1672-4cf8-8e78-f1fb3236997c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:53.359130 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96605b3-1672-4cf8-8e78-f1fb3236997c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:53.365241 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61450a4-c7b5-43c0-b41e-d6cec3d80d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:53.365967 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61450a4-c7b5-43c0-b41e-d6cec3d80d59 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:53.366457 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61450a4-c7b5-43c0-b41e-d6cec3d80d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:53.366735 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61450a4-c7b5-43c0-b41e-d6cec3d80d59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:53.367465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61450a4-c7b5-43c0-b41e-d6cec3d80d59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:05:53.367791 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61450a4-c7b5-43c0-b41e-d6cec3d80d59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:53.369448 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61450a4-c7b5-43c0-b41e-d6cec3d80d59 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:53.387611 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1265/2526] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:05:53.391464 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d37b7e-802a-41c3-936f-1646fc5f128c None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:53.402970 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d37b7e-802a-41c3-936f-1646fc5f128c None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:53.408205 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374a140-6fc5-4c5e-9643-6ed808ad1e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:53.408937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374a140-6fc5-4c5e-9643-6ed808ad1e60 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:53.409433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374a140-6fc5-4c5e-9643-6ed808ad1e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:53.409793 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374a140-6fc5-4c5e-9643-6ed808ad1e60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:53.410584 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374a140-6fc5-4c5e-9643-6ed808ad1e60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:53.412675 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374a140-6fc5-4c5e-9643-6ed808ad1e60 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:53.446984 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1262/2527] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:05:53 2022] POST /identity/v3/projects => generated 399 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:05:53.451269 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61450a4-c7b5-43c0-b41e-d6cec3d80d59 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:53.463124 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61450a4-c7b5-43c0-b41e-d6cec3d80d59 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:53.536877 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41ab5b-ce9b-425a-bd6c-dc1ea4da8f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:53.538534 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41ab5b-ce9b-425a-bd6c-dc1ea4da8f4f admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:53.539150 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41ab5b-ce9b-425a-bd6c-dc1ea4da8f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:53.539814 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41ab5b-ce9b-425a-bd6c-dc1ea4da8f4f admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:53.540525 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa41ab5b-ce9b-425a-bd6c-dc1ea4da8f4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:53.542458 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa41ab5b-ce9b-425a-bd6c-dc1ea4da8f4f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:53.567224 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1266/2528] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:05:53 2022] POST /identity/v3/users => generated 300 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:05:53.571745 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374a140-6fc5-4c5e-9643-6ed808ad1e60 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:53.583470 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374a140-6fc5-4c5e-9643-6ed808ad1e60 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:53.588250 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97798f5-a4f6-4d94-9309-82c40fbe285f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:53.588924 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97798f5-a4f6-4d94-9309-82c40fbe285f admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:53.589170 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97798f5-a4f6-4d94-9309-82c40fbe285f admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:53.589446 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97798f5-a4f6-4d94-9309-82c40fbe285f admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:53.590169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97798f5-a4f6-4d94-9309-82c40fbe285f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:53.591978 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97798f5-a4f6-4d94-9309-82c40fbe285f admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:53.603438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1263/2529] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:05:53.607771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41ab5b-ce9b-425a-bd6c-dc1ea4da8f4f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:53.622053 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41ab5b-ce9b-425a-bd6c-dc1ea4da8f4f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:53.627428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a9b73-1639-449a-bad0-9434f62d5ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:53.628160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a9b73-1639-449a-bad0-9434f62d5ea4 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:53.628452 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a9b73-1639-449a-bad0-9434f62d5ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:53.628731 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a9b73-1639-449a-bad0-9434f62d5ea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:53.629438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a9b73-1639-449a-bad0-9434f62d5ea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:53.631058 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a9b73-1639-449a-bad0-9434f62d5ea4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:53.646588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1267/2530] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:05:53.651018 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97798f5-a4f6-4d94-9309-82c40fbe285f admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:53.663188 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97798f5-a4f6-4d94-9309-82c40fbe285f admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:53.667925 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0766002-24c2-413d-802d-09cda00a5ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:53.668603 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0766002-24c2-413d-802d-09cda00a5ca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:53.668882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0766002-24c2-413d-802d-09cda00a5ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:53.669154 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0766002-24c2-413d-802d-09cda00a5ca2 admin admin] PATH_INFO: `/v3/projects/72fa534efa4f4744a66be5fbafe1e60a/users/3a7a296daf214a419c68a8cb1640cf94/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:53.688336 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0766002-24c2-413d-802d-09cda00a5ca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72fa534efa4f4744a66be5fbafe1e60a, user_id=3a7a296daf214a419c68a8cb1640cf94, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:53.691294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0766002-24c2-413d-802d-09cda00a5ca2 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:53.694378 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:53.711800 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1264/2531] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:05:53 2022] PUT /identity/v3/projects/72fa534efa4f4744a66be5fbafe1e60a/users/3a7a296daf214a419c68a8cb1640cf94/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:05:53.722581 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a069b6-1baa-472b-8af8-a5bbfd26cf6e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:53.722877 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a069b6-1baa-472b-8af8-a5bbfd26cf6e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:53.723154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a069b6-1baa-472b-8af8-a5bbfd26cf6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:53.755538 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:53.769131 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f9a069b6-1baa-472b-8af8-a5bbfd26cf6e None None] MFA Rules not processed for user `3a7a296daf214a419c68a8cb1640cf94`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:05:53.771684 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f9a069b6-1baa-472b-8af8-a5bbfd26cf6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:53.867571 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1268/2532] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:05:53 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:05:53.872227 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0766002-24c2-413d-802d-09cda00a5ca2 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:53.956718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0766002-24c2-413d-802d-09cda00a5ca2 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:53.966623 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459c964-0941-4caf-82e9-a522fdecb846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:53.967382 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459c964-0941-4caf-82e9-a522fdecb846 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:53.967628 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459c964-0941-4caf-82e9-a522fdecb846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:53.967913 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459c964-0941-4caf-82e9-a522fdecb846 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:53.968774 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5459c964-0941-4caf-82e9-a522fdecb846 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:53.970812 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5459c964-0941-4caf-82e9-a522fdecb846 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:54.012653 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1265/2533] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:05:53 2022] POST /identity/v3/projects => generated 397 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:05:54.013918 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a069b6-1baa-472b-8af8-a5bbfd26cf6e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:54.027475 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a069b6-1baa-472b-8af8-a5bbfd26cf6e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:54.129669 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f389537-2176-4ec4-b048-9a9c107f5b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:54.130033 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f389537-2176-4ec4-b048-9a9c107f5b0c admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:54.130236 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f389537-2176-4ec4-b048-9a9c107f5b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:54.130743 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f389537-2176-4ec4-b048-9a9c107f5b0c admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:54.131423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f389537-2176-4ec4-b048-9a9c107f5b0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:54.135048 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f389537-2176-4ec4-b048-9a9c107f5b0c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:54.164742 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1269/2534] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:05:54 2022] POST /identity/v3/users => generated 305 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:05:54.167962 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459c964-0941-4caf-82e9-a522fdecb846 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:54.178979 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459c964-0941-4caf-82e9-a522fdecb846 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:54.184352 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f1937-5049-4319-b147-b3ef38d7bc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:54.197824 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f1937-5049-4319-b147-b3ef38d7bc7d admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:54.198112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f1937-5049-4319-b147-b3ef38d7bc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:54.198379 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f1937-5049-4319-b147-b3ef38d7bc7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:54.199114 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f1937-5049-4319-b147-b3ef38d7bc7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:54.202385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f1937-5049-4319-b147-b3ef38d7bc7d admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:54.245775 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1266/2535] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:05:54.250147 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f389537-2176-4ec4-b048-9a9c107f5b0c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:54.262205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f389537-2176-4ec4-b048-9a9c107f5b0c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:54.268576 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57095f-689c-48a7-8473-b83b1434a3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:54.269469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57095f-689c-48a7-8473-b83b1434a3fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:54.269832 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57095f-689c-48a7-8473-b83b1434a3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:54.270219 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57095f-689c-48a7-8473-b83b1434a3fd admin admin] PATH_INFO: `/v3/projects/3e245d09e1044d5ba1ec76df54348ba1/users/902d932b7edd443ea360609892acb36e/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:54.288939 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f57095f-689c-48a7-8473-b83b1434a3fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e245d09e1044d5ba1ec76df54348ba1, user_id=902d932b7edd443ea360609892acb36e, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:54.291713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f57095f-689c-48a7-8473-b83b1434a3fd admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:54.313738 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1270/2536] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:05:54 2022] PUT /identity/v3/projects/3e245d09e1044d5ba1ec76df54348ba1/users/902d932b7edd443ea360609892acb36e/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:05:54.318443 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f1937-5049-4319-b147-b3ef38d7bc7d admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:54.338080 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f1937-5049-4319-b147-b3ef38d7bc7d admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:54.454081 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f446b27-62b5-4a97-97a8-cadff078bae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:54.454748 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f446b27-62b5-4a97-97a8-cadff078bae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:54.455032 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f446b27-62b5-4a97-97a8-cadff078bae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:54.455917 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f446b27-62b5-4a97-97a8-cadff078bae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:54.457020 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f446b27-62b5-4a97-97a8-cadff078bae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:54.459065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f446b27-62b5-4a97-97a8-cadff078bae5 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:54.470340 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1267/2537] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:05:54.474767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57095f-689c-48a7-8473-b83b1434a3fd admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:54.489575 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57095f-689c-48a7-8473-b83b1434a3fd admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:54.495323 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57c9e1-6e64-40d6-95b6-298e417d6d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:54.496342 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57c9e1-6e64-40d6-95b6-298e417d6d6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:54.496777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57c9e1-6e64-40d6-95b6-298e417d6d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:54.497226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57c9e1-6e64-40d6-95b6-298e417d6d6d admin admin] PATH_INFO: `/v3/projects/3e245d09e1044d5ba1ec76df54348ba1/users/902d932b7edd443ea360609892acb36e/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:54.500450 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57c9e1-6e64-40d6-95b6-298e417d6d6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e245d09e1044d5ba1ec76df54348ba1, user_id=902d932b7edd443ea360609892acb36e, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:54.503617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57c9e1-6e64-40d6-95b6-298e417d6d6d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:54.525008 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1271/2538] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:05:54 2022] PUT /identity/v3/projects/3e245d09e1044d5ba1ec76df54348ba1/users/902d932b7edd443ea360609892acb36e/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:05:54.529088 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff90f0b-f376-46ab-8c98-7ab097264fde None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:54.529088 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff90f0b-f376-46ab-8c98-7ab097264fde None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:54.529088 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff90f0b-f376-46ab-8c98-7ab097264fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:54.569265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-fff90f0b-f376-46ab-8c98-7ab097264fde None None] MFA Rules not processed for user `902d932b7edd443ea360609892acb36e`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:05:54.570893 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-fff90f0b-f376-46ab-8c98-7ab097264fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:54.633357 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:54.660687 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1268/2539] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:05:54 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:05:54.671977 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57c9e1-6e64-40d6-95b6-298e417d6d6d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:54.673797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1d57c9e1-6e64-40d6-95b6-298e417d6d6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:54.746137 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57c9e1-6e64-40d6-95b6-298e417d6d6d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:54.750183 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea482253-b56e-4e96-b910-f56e77422e22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:54.751021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea482253-b56e-4e96-b910-f56e77422e22 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:54.751447 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea482253-b56e-4e96-b910-f56e77422e22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:54.751897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea482253-b56e-4e96-b910-f56e77422e22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:54.762086 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:54.765405 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea482253-b56e-4e96-b910-f56e77422e22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:54.766941 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea482253-b56e-4e96-b910-f56e77422e22 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:54.780665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1272/2540] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:05:54 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:05:54.869001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff90f0b-f376-46ab-8c98-7ab097264fde None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:54.883904 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff90f0b-f376-46ab-8c98-7ab097264fde None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:54.887897 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a154a2-0991-4839-9075-92ca62be665d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:54.888588 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a154a2-0991-4839-9075-92ca62be665d service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:54.888993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a154a2-0991-4839-9075-92ca62be665d service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:54.889331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a154a2-0991-4839-9075-92ca62be665d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:54.967821 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a154a2-0991-4839-9075-92ca62be665d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:54.969418 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a154a2-0991-4839-9075-92ca62be665d service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:54.982426 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1269/2541] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:05:54 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:05:57.253603 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea482253-b56e-4e96-b910-f56e77422e22 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:57.257230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ea482253-b56e-4e96-b910-f56e77422e22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:57.346567 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea482253-b56e-4e96-b910-f56e77422e22 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:57.353984 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93f2f6-3619-4474-8198-0ee185b919eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:57.355204 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93f2f6-3619-4474-8198-0ee185b919eb service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:57.355204 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93f2f6-3619-4474-8198-0ee185b919eb service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:57.355417 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93f2f6-3619-4474-8198-0ee185b919eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:57.357447 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0a93f2f6-3619-4474-8198-0ee185b919eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:57.372330 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a93f2f6-3619-4474-8198-0ee185b919eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:57.377256 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a93f2f6-3619-4474-8198-0ee185b919eb service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:57.392517 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1273/2542] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:05:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:05:57.670155 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a154a2-0991-4839-9075-92ca62be665d service cinder] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:57.671930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-38a154a2-0991-4839-9075-92ca62be665d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:57.738918 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a154a2-0991-4839-9075-92ca62be665d service cinder] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:57.743485 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f90e5d6-f600-4ceb-b877-ed22c9cddda9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:57.744148 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90e5d6-f600-4ceb-b877-ed22c9cddda9 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:57.744581 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90e5d6-f600-4ceb-b877-ed22c9cddda9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:57.744860 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90e5d6-f600-4ceb-b877-ed22c9cddda9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:57.746718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-3f90e5d6-f600-4ceb-b877-ed22c9cddda9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:57.756869 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f90e5d6-f600-4ceb-b877-ed22c9cddda9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:57.758444 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f90e5d6-f600-4ceb-b877-ed22c9cddda9 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:57.767720 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1270/2543] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:05:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:05:59.208791 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c08bf-a515-4299-afd8-1ec19cabbab1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:59.209303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c08bf-a515-4299-afd8-1ec19cabbab1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:59.209303 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c08bf-a515-4299-afd8-1ec19cabbab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:59.231247 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-af9c08bf-a515-4299-afd8-1ec19cabbab1 None None] MFA Rules not processed for user `148ea5a9b7a74172b290696be99aa5c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:05:59.232525 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-af9c08bf-a515-4299-afd8-1ec19cabbab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:59.292407 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1274/2544] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:05:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:05:59.444803 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f90e5d6-f600-4ceb-b877-ed22c9cddda9 service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:59.512940 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f90e5d6-f600-4ceb-b877-ed22c9cddda9 service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:59.517741 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ef6c9-80f7-4481-809f-6e0a5efc034b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:59.518401 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ef6c9-80f7-4481-809f-6e0a5efc034b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:59.518688 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ef6c9-80f7-4481-809f-6e0a5efc034b admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:59.518965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ef6c9-80f7-4481-809f-6e0a5efc034b admin admin] PATH_INFO: `/v3/users/3a7a296daf214a419c68a8cb1640cf94` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:59.520202 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ef6c9-80f7-4481-809f-6e0a5efc034b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a7a296daf214a419c68a8cb1640cf94)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:59.522161 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ef6c9-80f7-4481-809f-6e0a5efc034b admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:59.561866 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:59.577310 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8c6ef6c9-80f7-4481-809f-6e0a5efc034b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a7a296daf214a419c68a8cb1640cf94'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:59.585606 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8c6ef6c9-80f7-4481-809f-6e0a5efc034b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a7a296daf214a419c68a8cb1640cf94'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:59.594657 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8c6ef6c9-80f7-4481-809f-6e0a5efc034b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a7a296daf214a419c68a8cb1640cf94'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:59.595186 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8c6ef6c9-80f7-4481-809f-6e0a5efc034b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a7a296daf214a419c68a8cb1640cf94'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:59.595880 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-8c6ef6c9-80f7-4481-809f-6e0a5efc034b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a7a296daf214a419c68a8cb1640cf94'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:05:59.618810 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1271/2545] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:05:59 2022] DELETE /identity/v3/users/3a7a296daf214a419c68a8cb1640cf94 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:05:59.623423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31536793-de22-454f-bf45-682bf2f391b9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:59.623744 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31536793-de22-454f-bf45-682bf2f391b9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:59.624025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31536793-de22-454f-bf45-682bf2f391b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:59.644227 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-31536793-de22-454f-bf45-682bf2f391b9 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:05:59.645461 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-31536793-de22-454f-bf45-682bf2f391b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:59.711582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1275/2546] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:05:59.721399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ef6c9-80f7-4481-809f-6e0a5efc034b admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:59.723351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-8c6ef6c9-80f7-4481-809f-6e0a5efc034b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:59.735691 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31536793-de22-454f-bf45-682bf2f391b9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:05:59.737210 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-31536793-de22-454f-bf45-682bf2f391b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:59.771700 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:05:59.785291 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ef6c9-80f7-4481-809f-6e0a5efc034b admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:59.789068 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d755871-f385-4b53-b64a-4e067d7d3b39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:59.789689 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d755871-f385-4b53-b64a-4e067d7d3b39 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:59.789966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d755871-f385-4b53-b64a-4e067d7d3b39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:59.790235 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d755871-f385-4b53-b64a-4e067d7d3b39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:59.798658 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31536793-de22-454f-bf45-682bf2f391b9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:05:59.799935 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d755871-f385-4b53-b64a-4e067d7d3b39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:59.801400 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d755871-f385-4b53-b64a-4e067d7d3b39 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:59.802371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74165774-7b6a-4602-af40-4552c6e49e7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:05:59.802999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74165774-7b6a-4602-af40-4552c6e49e7e service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:05:59.803286 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74165774-7b6a-4602-af40-4552c6e49e7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:05:59.803584 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74165774-7b6a-4602-af40-4552c6e49e7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:05:59.805351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-74165774-7b6a-4602-af40-4552c6e49e7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:05:59.810967 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1272/2547] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:05:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:05:59.816263 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74165774-7b6a-4602-af40-4552c6e49e7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:05:59.817840 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74165774-7b6a-4602-af40-4552c6e49e7e service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:05:59.827157 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1276/2548] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:05:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:06:00.084915 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d755871-f385-4b53-b64a-4e067d7d3b39 service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:00.086340 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-2d755871-f385-4b53-b64a-4e067d7d3b39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:00.101749 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d755871-f385-4b53-b64a-4e067d7d3b39 service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:00.107769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68d92b-6b83-442b-9e71-214a0faf213e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:00.108455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68d92b-6b83-442b-9e71-214a0faf213e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:00.108746 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68d92b-6b83-442b-9e71-214a0faf213e admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:00.109132 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68d92b-6b83-442b-9e71-214a0faf213e admin admin] PATH_INFO: `/v3/projects/72fa534efa4f4744a66be5fbafe1e60a` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:00.110382 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a68d92b-6b83-442b-9e71-214a0faf213e admin admin] RBAC: Authorizing `identity:delete_project(project_id=72fa534efa4f4744a66be5fbafe1e60a)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:00.112415 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a68d92b-6b83-442b-9e71-214a0faf213e admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:00.174953 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9a68d92b-6b83-442b-9e71-214a0faf213e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72fa534efa4f4744a66be5fbafe1e60a', 'request_id': 'req-9a68d92b-6b83-442b-9e71-214a0faf213e'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:00.187497 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9a68d92b-6b83-442b-9e71-214a0faf213e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72fa534efa4f4744a66be5fbafe1e60a', 'request_id': 'req-9a68d92b-6b83-442b-9e71-214a0faf213e'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:00.195966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9a68d92b-6b83-442b-9e71-214a0faf213e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72fa534efa4f4744a66be5fbafe1e60a', 'request_id': 'req-9a68d92b-6b83-442b-9e71-214a0faf213e'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:00.210031 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9a68d92b-6b83-442b-9e71-214a0faf213e admin admin] The token cache is being invalidate because project 72fa534efa4f4744a66be5fbafe1e60a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:06:00.210419 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9a68d92b-6b83-442b-9e71-214a0faf213e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:00.211540 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-9a68d92b-6b83-442b-9e71-214a0faf213e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:00.216557 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1273/2549] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:06:00 2022] DELETE /identity/v3/projects/72fa534efa4f4744a66be5fbafe1e60a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:06:00.218029 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74165774-7b6a-4602-af40-4552c6e49e7e service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:00.231540 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74165774-7b6a-4602-af40-4552c6e49e7e service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:00.284690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68d92b-6b83-442b-9e71-214a0faf213e admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:00.286825 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-9a68d92b-6b83-442b-9e71-214a0faf213e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:00.344221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f208e-0e97-40e0-a31c-d15c93c44a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:00.345199 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f208e-0e97-40e0-a31c-d15c93c44a01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:00.345485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f208e-0e97-40e0-a31c-d15c93c44a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:00.346083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f208e-0e97-40e0-a31c-d15c93c44a01 admin admin] PATH_INFO: `/v3/users/902d932b7edd443ea360609892acb36e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:00.347920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f208e-0e97-40e0-a31c-d15c93c44a01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=902d932b7edd443ea360609892acb36e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:00.350362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f208e-0e97-40e0-a31c-d15c93c44a01 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:00.379806 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:06:00.436935 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-520f208e-0e97-40e0-a31c-d15c93c44a01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '902d932b7edd443ea360609892acb36e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:00.440399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68d92b-6b83-442b-9e71-214a0faf213e admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:00.444618 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-520f208e-0e97-40e0-a31c-d15c93c44a01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '902d932b7edd443ea360609892acb36e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:00.445299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-520f208e-0e97-40e0-a31c-d15c93c44a01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '902d932b7edd443ea360609892acb36e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:00.447067 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d7073-7e54-486f-bff7-2a6ae3d1a469 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:00.447944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d7073-7e54-486f-bff7-2a6ae3d1a469 service nova] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:00.448228 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d7073-7e54-486f-bff7-2a6ae3d1a469 service nova] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:00.448500 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d7073-7e54-486f-bff7-2a6ae3d1a469 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:00.450572 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-217d7073-7e54-486f-bff7-2a6ae3d1a469 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:00.457214 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-520f208e-0e97-40e0-a31c-d15c93c44a01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '902d932b7edd443ea360609892acb36e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:00.457933 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-520f208e-0e97-40e0-a31c-d15c93c44a01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '902d932b7edd443ea360609892acb36e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:00.467047 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d7073-7e54-486f-bff7-2a6ae3d1a469 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:00.468706 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d7073-7e54-486f-bff7-2a6ae3d1a469 service nova] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:00.479389 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1277/2550] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:06:00 2022] DELETE /identity/v3/users/902d932b7edd443ea360609892acb36e => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:06:00.482545 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f208e-0e97-40e0-a31c-d15c93c44a01 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:00.482545 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-520f208e-0e97-40e0-a31c-d15c93c44a01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:00.487097 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1274/2551] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:06:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:06:00.539161 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:06:00.564214 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f208e-0e97-40e0-a31c-d15c93c44a01 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:00.569657 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc0a138-56c1-4da7-ac74-e8ae32e87cfb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5049dba7ffde47b8af3e6fd55b5703e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:00.570288 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0a138-56c1-4da7-ac74-e8ae32e87cfb service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:00.570569 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0a138-56c1-4da7-ac74-e8ae32e87cfb service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:00.570842 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0a138-56c1-4da7-ac74-e8ae32e87cfb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:00.572642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3fc0a138-56c1-4da7-ac74-e8ae32e87cfb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:00.584945 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc0a138-56c1-4da7-ac74-e8ae32e87cfb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:00.586536 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc0a138-56c1-4da7-ac74-e8ae32e87cfb service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:00.598062 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1278/2552] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:06:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:06:00.892773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d7073-7e54-486f-bff7-2a6ae3d1a469 service nova] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:00.897770 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-217d7073-7e54-486f-bff7-2a6ae3d1a469 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:00.917473 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d7073-7e54-486f-bff7-2a6ae3d1a469 service nova] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:00.997396 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6de5de-0a03-46e0-9223-fbaca99ddfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:00.998343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6de5de-0a03-46e0-9223-fbaca99ddfd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:00.998517 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6de5de-0a03-46e0-9223-fbaca99ddfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:00.998794 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6de5de-0a03-46e0-9223-fbaca99ddfd1 admin admin] PATH_INFO: `/v3/projects/3e245d09e1044d5ba1ec76df54348ba1` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:01.000236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6de5de-0a03-46e0-9223-fbaca99ddfd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e245d09e1044d5ba1ec76df54348ba1)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:01.002708 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6de5de-0a03-46e0-9223-fbaca99ddfd1 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:01.054025 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6a6de5de-0a03-46e0-9223-fbaca99ddfd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e245d09e1044d5ba1ec76df54348ba1', 'request_id': 'req-6a6de5de-0a03-46e0-9223-fbaca99ddfd1'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:01.063191 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6a6de5de-0a03-46e0-9223-fbaca99ddfd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e245d09e1044d5ba1ec76df54348ba1', 'request_id': 'req-6a6de5de-0a03-46e0-9223-fbaca99ddfd1'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:01.071664 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6a6de5de-0a03-46e0-9223-fbaca99ddfd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e245d09e1044d5ba1ec76df54348ba1', 'request_id': 'req-6a6de5de-0a03-46e0-9223-fbaca99ddfd1'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:01.084172 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6a6de5de-0a03-46e0-9223-fbaca99ddfd1 admin admin] The token cache is being invalidate because project 3e245d09e1044d5ba1ec76df54348ba1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:06:01.084525 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6a6de5de-0a03-46e0-9223-fbaca99ddfd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:01.085255 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-6a6de5de-0a03-46e0-9223-fbaca99ddfd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:01.086804 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1275/2553] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:06:00 2022] DELETE /identity/v3/projects/3e245d09e1044d5ba1ec76df54348ba1 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:06:01.091912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7156049-9105-48c0-8d0f-7a46f1ceef3b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:01.092205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7156049-9105-48c0-8d0f-7a46f1ceef3b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:01.092608 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7156049-9105-48c0-8d0f-7a46f1ceef3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:01.114674 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e7156049-9105-48c0-8d0f-7a46f1ceef3b None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:06:01.116042 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e7156049-9105-48c0-8d0f-7a46f1ceef3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:01.207781 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1279/2554] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:06:01.272461 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021059b-3522-4eee-88ba-2099941a079b None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:01.273651 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021059b-3522-4eee-88ba-2099941a079b None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:01.275232 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021059b-3522-4eee-88ba-2099941a079b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:01.301333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-3021059b-3522-4eee-88ba-2099941a079b None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:06:01.302577 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-3021059b-3522-4eee-88ba-2099941a079b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:01.310742 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1276/2555] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:06:01.315886 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7156049-9105-48c0-8d0f-7a46f1ceef3b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:01.326180 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7156049-9105-48c0-8d0f-7a46f1ceef3b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:01.331063 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7135889-9aeb-4ea0-ba4a-5bfc6f65be77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:01.331719 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7135889-9aeb-4ea0-ba4a-5bfc6f65be77 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:01.332006 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7135889-9aeb-4ea0-ba4a-5bfc6f65be77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:01.332428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7135889-9aeb-4ea0-ba4a-5bfc6f65be77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:01.332961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7135889-9aeb-4ea0-ba4a-5bfc6f65be77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:06:01.333306 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7135889-9aeb-4ea0-ba4a-5bfc6f65be77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:01.338020 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7135889-9aeb-4ea0-ba4a-5bfc6f65be77 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:01.347482 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1280/2556] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:06:01.352392 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021059b-3522-4eee-88ba-2099941a079b None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:01.362691 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021059b-3522-4eee-88ba-2099941a079b None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:01.367411 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbf5c0-4646-48fc-8af6-ab448bd0b3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:01.368062 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbf5c0-4646-48fc-8af6-ab448bd0b3a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:01.368345 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbf5c0-4646-48fc-8af6-ab448bd0b3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:01.368620 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbf5c0-4646-48fc-8af6-ab448bd0b3a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:01.369295 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fbf5c0-4646-48fc-8af6-ab448bd0b3a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:01.371224 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fbf5c0-4646-48fc-8af6-ab448bd0b3a4 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:01.404658 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1277/2557] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:06:01 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:06:01.405848 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7135889-9aeb-4ea0-ba4a-5bfc6f65be77 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:01.416156 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7135889-9aeb-4ea0-ba4a-5bfc6f65be77 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:01.484945 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1b058-40d7-413c-9825-a3a7fe6e70fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:01.485622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1b058-40d7-413c-9825-a3a7fe6e70fa admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:01.485963 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1b058-40d7-413c-9825-a3a7fe6e70fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:01.486267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1b058-40d7-413c-9825-a3a7fe6e70fa admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:01.486950 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1b058-40d7-413c-9825-a3a7fe6e70fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:01.489084 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1b058-40d7-413c-9825-a3a7fe6e70fa admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:01.515353 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1281/2558] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:06:01 2022] POST /identity/v3/users => generated 286 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:06:01.516222 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbf5c0-4646-48fc-8af6-ab448bd0b3a4 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:01.528024 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbf5c0-4646-48fc-8af6-ab448bd0b3a4 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:01.533053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ce685-2ff3-4eee-bd01-6a59ba27c01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:01.533704 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ce685-2ff3-4eee-bd01-6a59ba27c01a admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:01.534084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ce685-2ff3-4eee-bd01-6a59ba27c01a admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:01.534421 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ce685-2ff3-4eee-bd01-6a59ba27c01a admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:01.535208 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ce685-2ff3-4eee-bd01-6a59ba27c01a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:01.536935 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ce685-2ff3-4eee-bd01-6a59ba27c01a admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:01.553858 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1278/2559] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:06:01.555760 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1b058-40d7-413c-9825-a3a7fe6e70fa admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:01.570190 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1b058-40d7-413c-9825-a3a7fe6e70fa admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:01.577752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b5e65-d88e-480e-bc52-c333cea76da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:01.578924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b5e65-d88e-480e-bc52-c333cea76da2 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:01.579367 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b5e65-d88e-480e-bc52-c333cea76da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:01.579819 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b5e65-d88e-480e-bc52-c333cea76da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:01.580695 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b5e65-d88e-480e-bc52-c333cea76da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:01.582759 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b5e65-d88e-480e-bc52-c333cea76da2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:01.596900 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1282/2560] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:06:01.601472 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ce685-2ff3-4eee-bd01-6a59ba27c01a admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:01.612717 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ce685-2ff3-4eee-bd01-6a59ba27c01a admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:01.617522 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f19ba-c6ec-4968-8a61-09257f41aaf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:01.618199 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f19ba-c6ec-4968-8a61-09257f41aaf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:01.618478 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f19ba-c6ec-4968-8a61-09257f41aaf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:01.618757 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f19ba-c6ec-4968-8a61-09257f41aaf8 admin admin] PATH_INFO: `/v3/projects/a30ecee1cac84583bb0e1c74eb333314/users/ae6d25b14db34c6392999d0f7a929772/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:01.631319 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:06:01.639918 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981f19ba-c6ec-4968-8a61-09257f41aaf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a30ecee1cac84583bb0e1c74eb333314, user_id=ae6d25b14db34c6392999d0f7a929772, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:01.643424 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981f19ba-c6ec-4968-8a61-09257f41aaf8 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:01.659348 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1279/2561] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:06:01 2022] PUT /identity/v3/projects/a30ecee1cac84583bb0e1c74eb333314/users/ae6d25b14db34c6392999d0f7a929772/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:06:01.665487 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309c3e0-afb0-474e-9532-ae0bb4074ea9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:01.665924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309c3e0-afb0-474e-9532-ae0bb4074ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:01.666345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309c3e0-afb0-474e-9532-ae0bb4074ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:01.689947 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e309c3e0-afb0-474e-9532-ae0bb4074ea9 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:06:01.691485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e309c3e0-afb0-474e-9532-ae0bb4074ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:01.736174 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:06:01.760109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1283/2562] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:06:01.770667 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f19ba-c6ec-4968-8a61-09257f41aaf8 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:01.772509 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-981f19ba-c6ec-4968-8a61-09257f41aaf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:01.836580 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f19ba-c6ec-4968-8a61-09257f41aaf8 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:01.842091 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d514fb-127a-443a-a187-8749e674364b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:01.842728 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d514fb-127a-443a-a187-8749e674364b service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:01.843015 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d514fb-127a-443a-a187-8749e674364b service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:01.843295 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d514fb-127a-443a-a187-8749e674364b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:01.871804 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d514fb-127a-443a-a187-8749e674364b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:01.873401 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d514fb-127a-443a-a187-8749e674364b service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:01.898909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1280/2563] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:06:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:06:01.913639 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b45fd-d7fa-47ee-a472-624010ad1e22 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:01.913639 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b45fd-d7fa-47ee-a472-624010ad1e22 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:01.913997 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b45fd-d7fa-47ee-a472-624010ad1e22 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:01.916008 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1284/2564] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jul 26 05:06:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 05:06:01.921608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371666c-5e30-4edc-98ae-17532658e205 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:01.921910 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371666c-5e30-4edc-98ae-17532658e205 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:01.922184 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371666c-5e30-4edc-98ae-17532658e205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:01.943919 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-e371666c-5e30-4edc-98ae-17532658e205 None None] MFA Rules not processed for user `2faeed9002ee4154bc980c146cd06bcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:06:01.945306 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-e371666c-5e30-4edc-98ae-17532658e205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:02.027232 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1281/2565] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jul 26 05:06:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:06:02.039999 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b45fd-d7fa-47ee-a472-624010ad1e22 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:02.042629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d43b45fd-d7fa-47ee-a472-624010ad1e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:02.106777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b45fd-d7fa-47ee-a472-624010ad1e22 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:02.110587 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660dce46-84c2-423c-a041-069fc1d8623a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6946a788d2ef49ff873ca8e81990527e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:02.111446 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660dce46-84c2-423c-a041-069fc1d8623a service glance] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:02.111779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660dce46-84c2-423c-a041-069fc1d8623a service glance] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:02.112196 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660dce46-84c2-423c-a041-069fc1d8623a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:02.128881 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660dce46-84c2-423c-a041-069fc1d8623a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:02.130710 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660dce46-84c2-423c-a041-069fc1d8623a service glance] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:02.144610 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1285/2566] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:06:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:06:02.231046 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79b943-50fd-418b-8866-b1cb15862258 None None] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:02.231340 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79b943-50fd-418b-8866-b1cb15862258 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:02.231651 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79b943-50fd-418b-8866-b1cb15862258 None None] PATH_INFO: `/` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:02.233229 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1282/2567] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jul 26 05:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 05:06:02.238953 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2e377-4465-4e5c-85c4-8127ed80809a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:02.239381 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2e377-4465-4e5c-85c4-8127ed80809a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:02.239703 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2e377-4465-4e5c-85c4-8127ed80809a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:02.261424 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c5e2e377-4465-4e5c-85c4-8127ed80809a None None] MFA Rules not processed for user `dbdb8997891241e9b2e829feb8023f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:06:02.263648 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c5e2e377-4465-4e5c-85c4-8127ed80809a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:02.271230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1286/2568] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jul 26 05:06:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:06:02.286826 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79b943-50fd-418b-8866-b1cb15862258 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:02.288696 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-8d79b943-50fd-418b-8866-b1cb15862258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:02.304887 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79b943-50fd-418b-8866-b1cb15862258 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:02.308949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e47f8fa-68f0-41f8-a26c-aaf6a3ea63cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:02.310572 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47f8fa-68f0-41f8-a26c-aaf6a3ea63cd service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:02.311104 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47f8fa-68f0-41f8-a26c-aaf6a3ea63cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:02.311521 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47f8fa-68f0-41f8-a26c-aaf6a3ea63cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:02.323802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e47f8fa-68f0-41f8-a26c-aaf6a3ea63cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:02.325785 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e47f8fa-68f0-41f8-a26c-aaf6a3ea63cd service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:02.336475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1283/2569] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:06:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:06:02.441632 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9765c-09bf-44c8-aaad-24c3ef25b98c None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:02.442556 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9765c-09bf-44c8-aaad-24c3ef25b98c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:02.443021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9765c-09bf-44c8-aaad-24c3ef25b98c None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:02.445086 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1287/2570] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jul 26 05:06:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 05:06:02.452111 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0809161-53ca-4924-b259-96a6ecd167ce None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:02.452553 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0809161-53ca-4924-b259-96a6ecd167ce None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:02.452924 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0809161-53ca-4924-b259-96a6ecd167ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:02.475718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-f0809161-53ca-4924-b259-96a6ecd167ce None None] MFA Rules not processed for user `2faeed9002ee4154bc980c146cd06bcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:06:02.477045 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-f0809161-53ca-4924-b259-96a6ecd167ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:02.486798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1284/2571] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jul 26 05:06:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:06:02.500930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9765c-09bf-44c8-aaad-24c3ef25b98c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:02.518379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9765c-09bf-44c8-aaad-24c3ef25b98c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:02.523887 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be6851-adbf-424d-b259-4e7698fd06f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:02.525131 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be6851-adbf-424d-b259-4e7698fd06f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:02.525948 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be6851-adbf-424d-b259-4e7698fd06f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:02.526384 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be6851-adbf-424d-b259-4e7698fd06f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:02.544057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be6851-adbf-424d-b259-4e7698fd06f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:02.546995 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be6851-adbf-424d-b259-4e7698fd06f3 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:02.563025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1288/2572] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:06:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:06:03.287877 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9596514-c8b4-4e7e-9948-1b340d23699c None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:03.289600 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9596514-c8b4-4e7e-9948-1b340d23699c None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:03.290074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9596514-c8b4-4e7e-9948-1b340d23699c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:03.348159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-b9596514-c8b4-4e7e-9948-1b340d23699c None None] MFA Rules not processed for user `ae6d25b14db34c6392999d0f7a929772`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:06:03.350027 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-b9596514-c8b4-4e7e-9948-1b340d23699c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:03.404098 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:06:03.423996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1285/2573] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:06:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:06:03.438332 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be6851-adbf-424d-b259-4e7698fd06f3 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:03.440475 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c3be6851-adbf-424d-b259-4e7698fd06f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:03.551075 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be6851-adbf-424d-b259-4e7698fd06f3 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:03.554882 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ffc16-b541-40e6-b341-dfa3926a8c14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:03.555669 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ffc16-b541-40e6-b341-dfa3926a8c14 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:03.556046 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ffc16-b541-40e6-b341-dfa3926a8c14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:03.556413 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ffc16-b541-40e6-b341-dfa3926a8c14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:03.566278 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ffc16-b541-40e6-b341-dfa3926a8c14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:03.567990 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ffc16-b541-40e6-b341-dfa3926a8c14 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:03.579650 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1289/2574] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:06:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:06:05.138623 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33970196-310c-4bbd-a007-2547b318158d None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:05.139098 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33970196-310c-4bbd-a007-2547b318158d None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:05.139388 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33970196-310c-4bbd-a007-2547b318158d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:05.175850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-33970196-310c-4bbd-a007-2547b318158d None None] MFA Rules not processed for user `ae6d25b14db34c6392999d0f7a929772`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:06:05.177468 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-33970196-310c-4bbd-a007-2547b318158d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:05.194766 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1286/2575] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:06:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:06:05.946638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa98d1-f263-4499-a305-db3235336fd8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:05.947158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa98d1-f263-4499-a305-db3235336fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:05.947158 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa98d1-f263-4499-a305-db3235336fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:05.967542 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6afa98d1-f263-4499-a305-db3235336fd8 None None] MFA Rules not processed for user `3a8f3eb811ae40ad9cb64261f95a992e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:06:05.968867 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6afa98d1-f263-4499-a305-db3235336fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:06.026725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1290/2576] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jul 26 05:06:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:06:06.038995 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33970196-310c-4bbd-a007-2547b318158d None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:06.040510 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-33970196-310c-4bbd-a007-2547b318158d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:06.109992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33970196-310c-4bbd-a007-2547b318158d None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:06.115359 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f2f11-b181-4d76-b38b-c4151ff7ba86 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5218319aaa5424882714bffcbeee730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:06.116026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f2f11-b181-4d76-b38b-c4151ff7ba86 service swift] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:06.116300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f2f11-b181-4d76-b38b-c4151ff7ba86 service swift] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:06.116594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f2f11-b181-4d76-b38b-c4151ff7ba86 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:06.126371 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f2f11-b181-4d76-b38b-c4151ff7ba86 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:06.128787 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f2f11-b181-4d76-b38b-c4151ff7ba86 service swift] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:06.137660 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1287/2577] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jul 26 05:06:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 05:06:16.020908 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=93 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:06:16.134337 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=39 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:06:24.903182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dffb6a-790e-464a-8f93-30dd5d7c44fe None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:24.903182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dffb6a-790e-464a-8f93-30dd5d7c44fe None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:24.903778 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dffb6a-790e-464a-8f93-30dd5d7c44fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:24.923919 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f5dffb6a-790e-464a-8f93-30dd5d7c44fe None None] MFA Rules not processed for user `3a8f3eb811ae40ad9cb64261f95a992e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:06:24.925126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f5dffb6a-790e-464a-8f93-30dd5d7c44fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:24.932935 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1291/2578] 10.222.0.17 () {58 vars in 967 bytes} [Tue Jul 26 05:06:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:06:24.946569 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f2f11-b181-4d76-b38b-c4151ff7ba86 service swift] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:24.957174 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f2f11-b181-4d76-b38b-c4151ff7ba86 service swift] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:24.962956 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce6474-a5b9-4f3f-b521-2a803bd52d12 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5218319aaa5424882714bffcbeee730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:24.966651 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce6474-a5b9-4f3f-b521-2a803bd52d12 service swift] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:24.966927 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce6474-a5b9-4f3f-b521-2a803bd52d12 service swift] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:24.967200 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce6474-a5b9-4f3f-b521-2a803bd52d12 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:24.976585 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdce6474-a5b9-4f3f-b521-2a803bd52d12 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:24.978193 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdce6474-a5b9-4f3f-b521-2a803bd52d12 service swift] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:24.987672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1288/2579] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jul 26 05:06:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 05:06:27.869335 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dffb6a-790e-464a-8f93-30dd5d7c44fe None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:27.870866 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f5dffb6a-790e-464a-8f93-30dd5d7c44fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:27.888077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dffb6a-790e-464a-8f93-30dd5d7c44fe None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:27.894309 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac9b5f-034e-4e86-9c17-f24e7cc823c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:27.895144 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac9b5f-034e-4e86-9c17-f24e7cc823c7 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:27.895439 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac9b5f-034e-4e86-9c17-f24e7cc823c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:27.895939 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac9b5f-034e-4e86-9c17-f24e7cc823c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:27.897762 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-26ac9b5f-034e-4e86-9c17-f24e7cc823c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:27.911470 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac9b5f-034e-4e86-9c17-f24e7cc823c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:27.913094 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac9b5f-034e-4e86-9c17-f24e7cc823c7 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:27.928450 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1292/2580] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:06:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:06:31.227079 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce6474-a5b9-4f3f-b521-2a803bd52d12 service swift] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:31.238020 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce6474-a5b9-4f3f-b521-2a803bd52d12 service swift] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:31.242392 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d689f94-d724-44be-99e3-eb0ed4fa03ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:31.243022 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d689f94-d724-44be-99e3-eb0ed4fa03ac service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:31.243284 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d689f94-d724-44be-99e3-eb0ed4fa03ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:31.243541 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d689f94-d724-44be-99e3-eb0ed4fa03ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:31.245564 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-5d689f94-d724-44be-99e3-eb0ed4fa03ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:31.258296 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d689f94-d724-44be-99e3-eb0ed4fa03ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:31.259776 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d689f94-d724-44be-99e3-eb0ed4fa03ac service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:31.269749 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1289/2581] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:06:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:06:34.803522 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac9b5f-034e-4e86-9c17-f24e7cc823c7 service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:34.805187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-26ac9b5f-034e-4e86-9c17-f24e7cc823c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:34.821215 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac9b5f-034e-4e86-9c17-f24e7cc823c7 service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:34.826109 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33ef59-4e14-434d-983d-ad4fd160c422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:34.826918 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33ef59-4e14-434d-983d-ad4fd160c422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:34.827034 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33ef59-4e14-434d-983d-ad4fd160c422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:34.827370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33ef59-4e14-434d-983d-ad4fd160c422 admin admin] PATH_INFO: `/v3/users/39de5897bd2d4e2b91c05d8dda5827be` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:34.829319 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33ef59-4e14-434d-983d-ad4fd160c422 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39de5897bd2d4e2b91c05d8dda5827be)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:34.831200 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33ef59-4e14-434d-983d-ad4fd160c422 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:34.883189 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2f33ef59-4e14-434d-983d-ad4fd160c422 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39de5897bd2d4e2b91c05d8dda5827be'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:34.890647 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2f33ef59-4e14-434d-983d-ad4fd160c422 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39de5897bd2d4e2b91c05d8dda5827be'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:34.891508 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2f33ef59-4e14-434d-983d-ad4fd160c422 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39de5897bd2d4e2b91c05d8dda5827be'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:34.901364 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2f33ef59-4e14-434d-983d-ad4fd160c422 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39de5897bd2d4e2b91c05d8dda5827be'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:34.902140 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2f33ef59-4e14-434d-983d-ad4fd160c422 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39de5897bd2d4e2b91c05d8dda5827be'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:34.923283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1293/2582] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:06:34 2022] DELETE /identity/v3/users/39de5897bd2d4e2b91c05d8dda5827be => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:06:35.113938 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d689f94-d724-44be-99e3-eb0ed4fa03ac service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:35.131007 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d689f94-d724-44be-99e3-eb0ed4fa03ac service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:35.194786 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95141b-58c2-4f40-a57a-f78ad8634ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:35.195502 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95141b-58c2-4f40-a57a-f78ad8634ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:35.195850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95141b-58c2-4f40-a57a-f78ad8634ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:35.196132 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95141b-58c2-4f40-a57a-f78ad8634ec9 admin admin] PATH_INFO: `/v3/projects/9747c4b81e8844e989d70b8001a196b0` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:35.197327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da95141b-58c2-4f40-a57a-f78ad8634ec9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9747c4b81e8844e989d70b8001a196b0)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:35.199311 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da95141b-58c2-4f40-a57a-f78ad8634ec9 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:35.250919 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-da95141b-58c2-4f40-a57a-f78ad8634ec9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9747c4b81e8844e989d70b8001a196b0', 'request_id': 'req-da95141b-58c2-4f40-a57a-f78ad8634ec9'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:35.261349 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-da95141b-58c2-4f40-a57a-f78ad8634ec9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9747c4b81e8844e989d70b8001a196b0', 'request_id': 'req-da95141b-58c2-4f40-a57a-f78ad8634ec9'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:35.268471 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-da95141b-58c2-4f40-a57a-f78ad8634ec9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9747c4b81e8844e989d70b8001a196b0', 'request_id': 'req-da95141b-58c2-4f40-a57a-f78ad8634ec9'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:35.279947 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-da95141b-58c2-4f40-a57a-f78ad8634ec9 admin admin] The token cache is being invalidate because project 9747c4b81e8844e989d70b8001a196b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:06:35.280243 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-da95141b-58c2-4f40-a57a-f78ad8634ec9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:35.280926 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-da95141b-58c2-4f40-a57a-f78ad8634ec9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:35.282600 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1290/2583] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:06:35 2022] DELETE /identity/v3/projects/9747c4b81e8844e989d70b8001a196b0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:06:35.287891 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0cd8d-73ea-45a3-8bbd-50d43dfb0176 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:35.288176 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0cd8d-73ea-45a3-8bbd-50d43dfb0176 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:35.288512 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0cd8d-73ea-45a3-8bbd-50d43dfb0176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:35.309827 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a1f0cd8d-73ea-45a3-8bbd-50d43dfb0176 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:06:35.311501 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a1f0cd8d-73ea-45a3-8bbd-50d43dfb0176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:35.372944 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1294/2584] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:06:35.381222 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc918b9-ce32-43b9-9a8d-f1657fbe41be None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:35.381508 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc918b9-ce32-43b9-9a8d-f1657fbe41be None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:35.381789 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc918b9-ce32-43b9-9a8d-f1657fbe41be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:35.406352 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-9cc918b9-ce32-43b9-9a8d-f1657fbe41be None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:06:35.407656 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-9cc918b9-ce32-43b9-9a8d-f1657fbe41be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:35.416605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1291/2585] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:06:35.421043 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0cd8d-73ea-45a3-8bbd-50d43dfb0176 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:35.431684 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0cd8d-73ea-45a3-8bbd-50d43dfb0176 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:35.437628 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6222e2d-0d88-4de5-857f-60776e284880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:35.438385 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6222e2d-0d88-4de5-857f-60776e284880 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:35.438666 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6222e2d-0d88-4de5-857f-60776e284880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:35.438938 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6222e2d-0d88-4de5-857f-60776e284880 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:35.439670 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6222e2d-0d88-4de5-857f-60776e284880 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:06:35.439970 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6222e2d-0d88-4de5-857f-60776e284880 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:35.441597 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6222e2d-0d88-4de5-857f-60776e284880 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:35.453120 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1295/2586] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:06:35.457523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc918b9-ce32-43b9-9a8d-f1657fbe41be None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:35.466965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc918b9-ce32-43b9-9a8d-f1657fbe41be None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:35.472025 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df517be7-1b84-4f25-ab7d-1279e209b43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:35.472632 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df517be7-1b84-4f25-ab7d-1279e209b43b admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:35.472899 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df517be7-1b84-4f25-ab7d-1279e209b43b admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:35.473169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df517be7-1b84-4f25-ab7d-1279e209b43b admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:35.473826 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df517be7-1b84-4f25-ab7d-1279e209b43b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:35.475802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df517be7-1b84-4f25-ab7d-1279e209b43b admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:35.505435 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1292/2587] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:06:35 2022] POST /identity/v3/projects => generated 357 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:06:35.511568 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6222e2d-0d88-4de5-857f-60776e284880 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:35.522378 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6222e2d-0d88-4de5-857f-60776e284880 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:35.565202 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:06:35.586915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4faf5-c9ce-4228-96c0-6611fbbac479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:35.587573 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4faf5-c9ce-4228-96c0-6611fbbac479 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:35.587861 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4faf5-c9ce-4228-96c0-6611fbbac479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:35.588128 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4faf5-c9ce-4228-96c0-6611fbbac479 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:35.588799 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4faf5-c9ce-4228-96c0-6611fbbac479 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:35.590713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4faf5-c9ce-4228-96c0-6611fbbac479 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:35.614790 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1296/2588] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:06:35 2022] POST /identity/v3/users => generated 279 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:06:35.619328 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df517be7-1b84-4f25-ab7d-1279e209b43b admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:35.629934 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df517be7-1b84-4f25-ab7d-1279e209b43b admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:35.634871 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b8d84-30e4-4571-9e53-cf62b28d001a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:35.635478 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b8d84-30e4-4571-9e53-cf62b28d001a admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:35.635857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b8d84-30e4-4571-9e53-cf62b28d001a admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:35.636127 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b8d84-30e4-4571-9e53-cf62b28d001a admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:35.636827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b8d84-30e4-4571-9e53-cf62b28d001a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:35.638440 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b8d84-30e4-4571-9e53-cf62b28d001a admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:35.648660 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1293/2589] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:06:35.652934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4faf5-c9ce-4228-96c0-6611fbbac479 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:35.664269 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4faf5-c9ce-4228-96c0-6611fbbac479 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:35.669860 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211af946-065d-48ef-9232-91f808a9bbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:35.670506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211af946-065d-48ef-9232-91f808a9bbc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:35.670798 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211af946-065d-48ef-9232-91f808a9bbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:35.671142 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211af946-065d-48ef-9232-91f808a9bbc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:35.671811 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211af946-065d-48ef-9232-91f808a9bbc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:35.673430 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211af946-065d-48ef-9232-91f808a9bbc6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:35.683939 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1297/2590] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:06:35.688649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b8d84-30e4-4571-9e53-cf62b28d001a admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:35.698849 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b8d84-30e4-4571-9e53-cf62b28d001a admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:35.703721 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd30c6-df25-44fb-a5f3-b9f3679ef76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:35.704320 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd30c6-df25-44fb-a5f3-b9f3679ef76f admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:35.704600 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd30c6-df25-44fb-a5f3-b9f3679ef76f admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:35.704873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd30c6-df25-44fb-a5f3-b9f3679ef76f admin admin] PATH_INFO: `/v3/projects/72291247e36c443aaded96abab1e8b2e/users/571f2bb3fdf0485babfcbf96226864ba/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:35.719375 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fd30c6-df25-44fb-a5f3-b9f3679ef76f admin admin] RBAC: Authorizing `identity:create_grant(project_id=72291247e36c443aaded96abab1e8b2e, user_id=571f2bb3fdf0485babfcbf96226864ba, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:35.722078 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fd30c6-df25-44fb-a5f3-b9f3679ef76f admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:35.732101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:06:35.738538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1294/2591] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:06:35 2022] PUT /identity/v3/projects/72291247e36c443aaded96abab1e8b2e/users/571f2bb3fdf0485babfcbf96226864ba/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:06:35.755374 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296440d-47f1-4275-824a-857b11403a4f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:35.755730 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296440d-47f1-4275-824a-857b11403a4f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:35.756061 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296440d-47f1-4275-824a-857b11403a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:35.786139 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6296440d-47f1-4275-824a-857b11403a4f None None] MFA Rules not processed for user `571f2bb3fdf0485babfcbf96226864ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:06:35.787377 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6296440d-47f1-4275-824a-857b11403a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:35.854969 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1298/2592] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:06:35 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:06:35.859847 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd30c6-df25-44fb-a5f3-b9f3679ef76f admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:35.939019 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd30c6-df25-44fb-a5f3-b9f3679ef76f admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:35.946299 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5ecf6-3010-48c6-9045-e9af4f7812fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:35.946932 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5ecf6-3010-48c6-9045-e9af4f7812fe admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:35.947736 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5ecf6-3010-48c6-9045-e9af4f7812fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:35.948022 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5ecf6-3010-48c6-9045-e9af4f7812fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:35.948729 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff5ecf6-3010-48c6-9045-e9af4f7812fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:35.952275 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff5ecf6-3010-48c6-9045-e9af4f7812fe admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:35.991038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1295/2593] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:06:35 2022] POST /identity/v3/projects => generated 357 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:06:35.995315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296440d-47f1-4275-824a-857b11403a4f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:36.006101 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296440d-47f1-4275-824a-857b11403a4f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:36.062525 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30079c33-d276-4902-a5db-3dbdc8840fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:36.063157 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30079c33-d276-4902-a5db-3dbdc8840fed admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:36.063441 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30079c33-d276-4902-a5db-3dbdc8840fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:36.063752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30079c33-d276-4902-a5db-3dbdc8840fed admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:36.064431 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30079c33-d276-4902-a5db-3dbdc8840fed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:36.066356 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30079c33-d276-4902-a5db-3dbdc8840fed admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:36.092082 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1299/2594] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:06:35 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:06:36.096177 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5ecf6-3010-48c6-9045-e9af4f7812fe admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:36.106803 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5ecf6-3010-48c6-9045-e9af4f7812fe admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:36.112333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028b088e-1376-46a7-883a-b5340876df4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:36.112948 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b088e-1376-46a7-883a-b5340876df4e admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:36.113227 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b088e-1376-46a7-883a-b5340876df4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:36.113496 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b088e-1376-46a7-883a-b5340876df4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:36.114195 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b088e-1376-46a7-883a-b5340876df4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:36.115871 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b088e-1376-46a7-883a-b5340876df4e admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:36.131263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1296/2595] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:06:36.132723 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30079c33-d276-4902-a5db-3dbdc8840fed admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:36.144121 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30079c33-d276-4902-a5db-3dbdc8840fed admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:36.149554 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12575e46-5c22-4273-ae88-7055ddef04c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:36.150271 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12575e46-5c22-4273-ae88-7055ddef04c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:36.150574 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12575e46-5c22-4273-ae88-7055ddef04c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:36.150842 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12575e46-5c22-4273-ae88-7055ddef04c3 admin admin] PATH_INFO: `/v3/projects/6565d92856c24b4c8c3df77fa6f6f9fb/users/a3b4f78afd9d4e6eb1e2785ef69c54b2/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:36.169397 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f04732-657d-4d97-9eff-f2e531ed9a6d None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:36.170024 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f04732-657d-4d97-9eff-f2e531ed9a6d None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:36.170149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f04732-657d-4d97-9eff-f2e531ed9a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:36.174160 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12575e46-5c22-4273-ae88-7055ddef04c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6565d92856c24b4c8c3df77fa6f6f9fb, user_id=a3b4f78afd9d4e6eb1e2785ef69c54b2, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:36.177098 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12575e46-5c22-4273-ae88-7055ddef04c3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:36.190205 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-59f04732-657d-4d97-9eff-f2e531ed9a6d None None] MFA Rules not processed for user `ae6d25b14db34c6392999d0f7a929772`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:06:36.191399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-59f04732-657d-4d97-9eff-f2e531ed9a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:36.192053 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1300/2596] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:06:36 2022] PUT /identity/v3/projects/6565d92856c24b4c8c3df77fa6f6f9fb/users/a3b4f78afd9d4e6eb1e2785ef69c54b2/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:06:36.196144 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12575e46-5c22-4273-ae88-7055ddef04c3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:36.209166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:06:36.249719 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1297/2597] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:06:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:06:36.268022 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12575e46-5c22-4273-ae88-7055ddef04c3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:36.273972 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4762dc1-39f5-4ff1-ba30-3d7225d7d547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:36.274617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4762dc1-39f5-4ff1-ba30-3d7225d7d547 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:36.274900 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4762dc1-39f5-4ff1-ba30-3d7225d7d547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:36.275176 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4762dc1-39f5-4ff1-ba30-3d7225d7d547 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:36.275921 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4762dc1-39f5-4ff1-ba30-3d7225d7d547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:36.277521 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4762dc1-39f5-4ff1-ba30-3d7225d7d547 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:36.288779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1301/2598] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:06:36.293321 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f04732-657d-4d97-9eff-f2e531ed9a6d None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:36.305132 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f04732-657d-4d97-9eff-f2e531ed9a6d None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:36.310338 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790bd76-b2a8-4824-a930-6879f974a067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:36.311394 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790bd76-b2a8-4824-a930-6879f974a067 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:36.311697 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790bd76-b2a8-4824-a930-6879f974a067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:36.311977 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790bd76-b2a8-4824-a930-6879f974a067 admin admin] PATH_INFO: `/v3/projects/6565d92856c24b4c8c3df77fa6f6f9fb/users/a3b4f78afd9d4e6eb1e2785ef69c54b2/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:36.313955 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f790bd76-b2a8-4824-a930-6879f974a067 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6565d92856c24b4c8c3df77fa6f6f9fb, user_id=a3b4f78afd9d4e6eb1e2785ef69c54b2, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:36.316689 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f790bd76-b2a8-4824-a930-6879f974a067 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:36.331110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1298/2599] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:06:36 2022] PUT /identity/v3/projects/6565d92856c24b4c8c3df77fa6f6f9fb/users/a3b4f78afd9d4e6eb1e2785ef69c54b2/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:06:36.342193 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f95a4e-c59b-4dc7-8b38-5e7c163d96b4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:36.342492 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f95a4e-c59b-4dc7-8b38-5e7c163d96b4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:36.342798 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f95a4e-c59b-4dc7-8b38-5e7c163d96b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:36.371996 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f9f95a4e-c59b-4dc7-8b38-5e7c163d96b4 None None] MFA Rules not processed for user `a3b4f78afd9d4e6eb1e2785ef69c54b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:06:36.373288 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f9f95a4e-c59b-4dc7-8b38-5e7c163d96b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:36.449087 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1302/2600] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:06:36 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:06:36.460927 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790bd76-b2a8-4824-a930-6879f974a067 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:36.463026 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-f790bd76-b2a8-4824-a930-6879f974a067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:36.528332 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790bd76-b2a8-4824-a930-6879f974a067 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:36.532173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8dc24-47de-483b-b6be-93d74c6554f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:36.533432 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8dc24-47de-483b-b6be-93d74c6554f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:36.533432 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8dc24-47de-483b-b6be-93d74c6554f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:36.533432 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8dc24-47de-483b-b6be-93d74c6554f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:36.544465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c8dc24-47de-483b-b6be-93d74c6554f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:36.545964 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c8dc24-47de-483b-b6be-93d74c6554f2 service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:36.556841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1299/2601] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:06:36 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:06:36.645704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f95a4e-c59b-4dc7-8b38-5e7c163d96b4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:36.656451 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f95a4e-c59b-4dc7-8b38-5e7c163d96b4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:36.660426 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f9b79-f8a0-4766-bf14-777651138051 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:36.661077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f9b79-f8a0-4766-bf14-777651138051 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:36.661361 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f9b79-f8a0-4766-bf14-777651138051 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:36.661633 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f9b79-f8a0-4766-bf14-777651138051 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:36.723221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f9b79-f8a0-4766-bf14-777651138051 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:36.724747 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f9b79-f8a0-4766-bf14-777651138051 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:36.734070 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1303/2602] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:06:36 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:06:46.551820 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=94,keystone:INSERT=2 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:06:46.729654 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=85 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:06:51.940457 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8dc24-47de-483b-b6be-93d74c6554f2 service cinder] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:52.010884 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8dc24-47de-483b-b6be-93d74c6554f2 service cinder] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:52.015797 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba2965-6828-4696-b525-bf6951119dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:52.016460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba2965-6828-4696-b525-bf6951119dc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:52.016744 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba2965-6828-4696-b525-bf6951119dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:52.017007 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba2965-6828-4696-b525-bf6951119dc6 admin admin] PATH_INFO: `/v3/users/571f2bb3fdf0485babfcbf96226864ba` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:52.018243 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba2965-6828-4696-b525-bf6951119dc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=571f2bb3fdf0485babfcbf96226864ba)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:52.020246 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba2965-6828-4696-b525-bf6951119dc6 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:52.074480 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-61ba2965-6828-4696-b525-bf6951119dc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '571f2bb3fdf0485babfcbf96226864ba'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:52.081763 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-61ba2965-6828-4696-b525-bf6951119dc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '571f2bb3fdf0485babfcbf96226864ba'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:52.090537 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-61ba2965-6828-4696-b525-bf6951119dc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '571f2bb3fdf0485babfcbf96226864ba'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:52.091182 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-61ba2965-6828-4696-b525-bf6951119dc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '571f2bb3fdf0485babfcbf96226864ba'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:52.091796 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-61ba2965-6828-4696-b525-bf6951119dc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '571f2bb3fdf0485babfcbf96226864ba'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:52.109616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1300/2603] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:06:51 2022] DELETE /identity/v3/users/571f2bb3fdf0485babfcbf96226864ba => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:06:52.114350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec202ea-43b1-40a7-9920-22c5f4ee3584 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:52.114856 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec202ea-43b1-40a7-9920-22c5f4ee3584 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:52.114952 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec202ea-43b1-40a7-9920-22c5f4ee3584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:52.135141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-aec202ea-43b1-40a7-9920-22c5f4ee3584 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:06:52.136494 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-aec202ea-43b1-40a7-9920-22c5f4ee3584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:52.203359 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1304/2604] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:06:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:06:52.214256 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba2965-6828-4696-b525-bf6951119dc6 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:52.215845 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-61ba2965-6828-4696-b525-bf6951119dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:52.282410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba2965-6828-4696-b525-bf6951119dc6 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:52.286210 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ffb998-2b90-49b6-84bb-6cc90140f091 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:52.286845 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffb998-2b90-49b6-84bb-6cc90140f091 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:52.287121 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffb998-2b90-49b6-84bb-6cc90140f091 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:52.287408 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffb998-2b90-49b6-84bb-6cc90140f091 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:52.299651 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ffb998-2b90-49b6-84bb-6cc90140f091 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:52.301753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ffb998-2b90-49b6-84bb-6cc90140f091 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:52.313245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1301/2605] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:06:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:06:52.600390 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec202ea-43b1-40a7-9920-22c5f4ee3584 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:52.601830 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-aec202ea-43b1-40a7-9920-22c5f4ee3584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:52.614641 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec202ea-43b1-40a7-9920-22c5f4ee3584 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:52.620466 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22897d-40a6-484d-b069-85b7f3c0463f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:52.621996 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22897d-40a6-484d-b069-85b7f3c0463f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:52.622292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22897d-40a6-484d-b069-85b7f3c0463f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:52.622566 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22897d-40a6-484d-b069-85b7f3c0463f admin admin] PATH_INFO: `/v3/projects/72291247e36c443aaded96abab1e8b2e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:52.623938 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af22897d-40a6-484d-b069-85b7f3c0463f admin admin] RBAC: Authorizing `identity:delete_project(project_id=72291247e36c443aaded96abab1e8b2e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:52.626021 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af22897d-40a6-484d-b069-85b7f3c0463f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:52.685909 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-af22897d-40a6-484d-b069-85b7f3c0463f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72291247e36c443aaded96abab1e8b2e', 'request_id': 'req-af22897d-40a6-484d-b069-85b7f3c0463f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:52.692797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-af22897d-40a6-484d-b069-85b7f3c0463f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72291247e36c443aaded96abab1e8b2e', 'request_id': 'req-af22897d-40a6-484d-b069-85b7f3c0463f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:52.702485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-af22897d-40a6-484d-b069-85b7f3c0463f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72291247e36c443aaded96abab1e8b2e', 'request_id': 'req-af22897d-40a6-484d-b069-85b7f3c0463f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:52.713243 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-af22897d-40a6-484d-b069-85b7f3c0463f admin admin] The token cache is being invalidate because project 72291247e36c443aaded96abab1e8b2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:06:52.713532 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-af22897d-40a6-484d-b069-85b7f3c0463f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:52.714196 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-af22897d-40a6-484d-b069-85b7f3c0463f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:52.715815 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1305/2606] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:06:52 2022] DELETE /identity/v3/projects/72291247e36c443aaded96abab1e8b2e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:06:52.721751 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ffb998-2b90-49b6-84bb-6cc90140f091 service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:52.734357 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ffb998-2b90-49b6-84bb-6cc90140f091 service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:52.761831 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:06:52.805869 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40350e28-0e7e-4796-af66-cdeea73fbff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:52.806440 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40350e28-0e7e-4796-af66-cdeea73fbff5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:52.806727 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40350e28-0e7e-4796-af66-cdeea73fbff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:52.807001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40350e28-0e7e-4796-af66-cdeea73fbff5 admin admin] PATH_INFO: `/v3/users/a3b4f78afd9d4e6eb1e2785ef69c54b2` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:52.808205 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40350e28-0e7e-4796-af66-cdeea73fbff5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3b4f78afd9d4e6eb1e2785ef69c54b2)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:52.810523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40350e28-0e7e-4796-af66-cdeea73fbff5 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:52.860911 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-40350e28-0e7e-4796-af66-cdeea73fbff5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3b4f78afd9d4e6eb1e2785ef69c54b2'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:52.867685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-40350e28-0e7e-4796-af66-cdeea73fbff5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3b4f78afd9d4e6eb1e2785ef69c54b2'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:52.876523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-40350e28-0e7e-4796-af66-cdeea73fbff5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3b4f78afd9d4e6eb1e2785ef69c54b2'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:52.877157 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-40350e28-0e7e-4796-af66-cdeea73fbff5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3b4f78afd9d4e6eb1e2785ef69c54b2'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:52.877870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-40350e28-0e7e-4796-af66-cdeea73fbff5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3b4f78afd9d4e6eb1e2785ef69c54b2'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:52.896315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1302/2607] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:06:52 2022] DELETE /identity/v3/users/a3b4f78afd9d4e6eb1e2785ef69c54b2 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:06:53.184018 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22897d-40a6-484d-b069-85b7f3c0463f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:53.186613 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-af22897d-40a6-484d-b069-85b7f3c0463f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:53.201891 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22897d-40a6-484d-b069-85b7f3c0463f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:53.268257 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e1393-c021-465b-9943-5ece5e34648a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:53.268930 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e1393-c021-465b-9943-5ece5e34648a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:53.269214 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e1393-c021-465b-9943-5ece5e34648a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:53.269488 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e1393-c021-465b-9943-5ece5e34648a admin admin] PATH_INFO: `/v3/projects/6565d92856c24b4c8c3df77fa6f6f9fb` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:53.270801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e1393-c021-465b-9943-5ece5e34648a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6565d92856c24b4c8c3df77fa6f6f9fb)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:53.272767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e1393-c021-465b-9943-5ece5e34648a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:53.309101 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:06:53.326449 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ba8e1393-c021-465b-9943-5ece5e34648a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6565d92856c24b4c8c3df77fa6f6f9fb', 'request_id': 'req-ba8e1393-c021-465b-9943-5ece5e34648a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:53.333191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ba8e1393-c021-465b-9943-5ece5e34648a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6565d92856c24b4c8c3df77fa6f6f9fb', 'request_id': 'req-ba8e1393-c021-465b-9943-5ece5e34648a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:53.343367 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ba8e1393-c021-465b-9943-5ece5e34648a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6565d92856c24b4c8c3df77fa6f6f9fb', 'request_id': 'req-ba8e1393-c021-465b-9943-5ece5e34648a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:53.355497 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ba8e1393-c021-465b-9943-5ece5e34648a admin admin] The token cache is being invalidate because project 6565d92856c24b4c8c3df77fa6f6f9fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:06:53.355907 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ba8e1393-c021-465b-9943-5ece5e34648a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:53.356460 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ba8e1393-c021-465b-9943-5ece5e34648a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:53.358064 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1306/2608] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:06:53 2022] DELETE /identity/v3/projects/6565d92856c24b4c8c3df77fa6f6f9fb => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:06:53.363225 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982de9f4-9106-41f4-985e-b31a4996e7e6 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:53.363768 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982de9f4-9106-41f4-985e-b31a4996e7e6 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:53.363973 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982de9f4-9106-41f4-985e-b31a4996e7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:53.383416 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-982de9f4-9106-41f4-985e-b31a4996e7e6 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:06:53.384673 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-982de9f4-9106-41f4-985e-b31a4996e7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:53.449356 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1303/2609] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:06:53.458910 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63e15f-3fcf-4acf-9f89-2ff9588ec588 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:53.459201 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63e15f-3fcf-4acf-9f89-2ff9588ec588 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:53.459478 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63e15f-3fcf-4acf-9f89-2ff9588ec588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:53.483060 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4f63e15f-3fcf-4acf-9f89-2ff9588ec588 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:06:53.484329 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4f63e15f-3fcf-4acf-9f89-2ff9588ec588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:53.492470 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1307/2610] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:06:53.497569 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982de9f4-9106-41f4-985e-b31a4996e7e6 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:53.507462 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982de9f4-9106-41f4-985e-b31a4996e7e6 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:53.512265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcdf048-5a73-47c4-860b-504e5149f0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:53.512886 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcdf048-5a73-47c4-860b-504e5149f0bf admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:53.513177 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcdf048-5a73-47c4-860b-504e5149f0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:53.513449 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcdf048-5a73-47c4-860b-504e5149f0bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:53.514163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcdf048-5a73-47c4-860b-504e5149f0bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:06:53.514469 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcdf048-5a73-47c4-860b-504e5149f0bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:53.516125 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcdf048-5a73-47c4-860b-504e5149f0bf admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:53.532864 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1304/2611] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:06:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:06:53.534083 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63e15f-3fcf-4acf-9f89-2ff9588ec588 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:53.543912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63e15f-3fcf-4acf-9f89-2ff9588ec588 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:53.548683 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ac7ba-6b71-4cb5-bd88-6a15bef321f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:53.549348 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ac7ba-6b71-4cb5-bd88-6a15bef321f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:53.549635 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ac7ba-6b71-4cb5-bd88-6a15bef321f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:53.549916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ac7ba-6b71-4cb5-bd88-6a15bef321f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:53.550622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ac7ba-6b71-4cb5-bd88-6a15bef321f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:53.552581 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ac7ba-6b71-4cb5-bd88-6a15bef321f9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:53.581998 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1308/2612] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:06:53 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:06:53.585950 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcdf048-5a73-47c4-860b-504e5149f0bf admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:53.596296 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcdf048-5a73-47c4-860b-504e5149f0bf admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:53.627319 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:06:53.662128 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe9e0f-a861-4883-8874-b09180e1f623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:53.662796 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe9e0f-a861-4883-8874-b09180e1f623 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:53.663078 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe9e0f-a861-4883-8874-b09180e1f623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:53.663384 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe9e0f-a861-4883-8874-b09180e1f623 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:53.664070 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe9e0f-a861-4883-8874-b09180e1f623 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:53.666008 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe9e0f-a861-4883-8874-b09180e1f623 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:53.689742 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1305/2613] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:06:53 2022] POST /identity/v3/users => generated 291 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:06:53.693818 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ac7ba-6b71-4cb5-bd88-6a15bef321f9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:53.704322 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ac7ba-6b71-4cb5-bd88-6a15bef321f9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:53.710075 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a6365-0e76-44c1-bc4e-0731c613d88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:53.710703 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a6365-0e76-44c1-bc4e-0731c613d88c admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:53.710983 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a6365-0e76-44c1-bc4e-0731c613d88c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:53.711321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a6365-0e76-44c1-bc4e-0731c613d88c admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:53.712056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a6365-0e76-44c1-bc4e-0731c613d88c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:53.713645 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a6365-0e76-44c1-bc4e-0731c613d88c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:53.725216 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1309/2614] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:06:53.729812 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe9e0f-a861-4883-8874-b09180e1f623 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:53.740937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe9e0f-a861-4883-8874-b09180e1f623 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:53.746132 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e1869-a29c-42d3-8d9d-2d39cd5e6c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:53.746947 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e1869-a29c-42d3-8d9d-2d39cd5e6c45 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:53.747234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e1869-a29c-42d3-8d9d-2d39cd5e6c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:53.747512 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e1869-a29c-42d3-8d9d-2d39cd5e6c45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:53.748346 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e1869-a29c-42d3-8d9d-2d39cd5e6c45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:53.750001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e1869-a29c-42d3-8d9d-2d39cd5e6c45 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:53.764552 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1306/2615] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:06:53.766730 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a6365-0e76-44c1-bc4e-0731c613d88c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:53.777013 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a6365-0e76-44c1-bc4e-0731c613d88c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:53.782484 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d17650-edb1-4d9c-bee4-edeba4e5e60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:53.783173 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d17650-edb1-4d9c-bee4-edeba4e5e60f admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:53.783460 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d17650-edb1-4d9c-bee4-edeba4e5e60f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:53.783777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d17650-edb1-4d9c-bee4-edeba4e5e60f admin admin] PATH_INFO: `/v3/projects/6d2a52aa395a41c1b3ab7eb4b2ba7f54/users/cb73e14e2f0f46e085b24936f94be45d/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:53.799906 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d17650-edb1-4d9c-bee4-edeba4e5e60f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d2a52aa395a41c1b3ab7eb4b2ba7f54, user_id=cb73e14e2f0f46e085b24936f94be45d, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:53.802538 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d17650-edb1-4d9c-bee4-edeba4e5e60f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:53.819816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1310/2616] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:06:53 2022] PUT /identity/v3/projects/6d2a52aa395a41c1b3ab7eb4b2ba7f54/users/cb73e14e2f0f46e085b24936f94be45d/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:06:53.830727 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e85add-3047-4759-9eb0-448d4ff2c914 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:53.831016 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e85add-3047-4759-9eb0-448d4ff2c914 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:53.831301 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e85add-3047-4759-9eb0-448d4ff2c914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:53.862453 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-a8e85add-3047-4759-9eb0-448d4ff2c914 None None] MFA Rules not processed for user `cb73e14e2f0f46e085b24936f94be45d`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:06:53.863723 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a8e85add-3047-4759-9eb0-448d4ff2c914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:53.929750 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1307/2617] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:06:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:06:53.935499 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d17650-edb1-4d9c-bee4-edeba4e5e60f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:54.002332 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d17650-edb1-4d9c-bee4-edeba4e5e60f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:54.007283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a838b-e701-4445-bb4d-700f2ebaef0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:54.007950 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a838b-e701-4445-bb4d-700f2ebaef0a admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:54.008229 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a838b-e701-4445-bb4d-700f2ebaef0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:54.008502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a838b-e701-4445-bb4d-700f2ebaef0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:54.009164 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a838b-e701-4445-bb4d-700f2ebaef0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:54.011052 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a838b-e701-4445-bb4d-700f2ebaef0a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:54.037798 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1311/2618] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:06:53 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:06:54.042162 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e85add-3047-4759-9eb0-448d4ff2c914 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:54.053791 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e85add-3047-4759-9eb0-448d4ff2c914 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:54.113490 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e32770-7305-44bf-a136-1a8df33abbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:54.114025 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e32770-7305-44bf-a136-1a8df33abbef admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:54.114306 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e32770-7305-44bf-a136-1a8df33abbef admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:54.114577 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e32770-7305-44bf-a136-1a8df33abbef admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:54.115237 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e32770-7305-44bf-a136-1a8df33abbef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:54.117217 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e32770-7305-44bf-a136-1a8df33abbef admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:54.139526 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1308/2619] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:06:54 2022] POST /identity/v3/users => generated 298 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:06:54.144849 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a838b-e701-4445-bb4d-700f2ebaef0a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:54.155652 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a838b-e701-4445-bb4d-700f2ebaef0a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:54.161047 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7065b2e-580e-4fec-9060-b17331e254ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:54.161658 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7065b2e-580e-4fec-9060-b17331e254ee admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:54.161993 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7065b2e-580e-4fec-9060-b17331e254ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:54.162338 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7065b2e-580e-4fec-9060-b17331e254ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:54.163231 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7065b2e-580e-4fec-9060-b17331e254ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:54.165182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7065b2e-580e-4fec-9060-b17331e254ee admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:54.176625 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1312/2620] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:06:54.180982 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e32770-7305-44bf-a136-1a8df33abbef admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:54.191824 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e32770-7305-44bf-a136-1a8df33abbef admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:54.196991 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091519e-516d-4678-8f59-563834651766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:54.197652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091519e-516d-4678-8f59-563834651766 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:54.197938 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091519e-516d-4678-8f59-563834651766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:54.198263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091519e-516d-4678-8f59-563834651766 admin admin] PATH_INFO: `/v3/projects/dd0c79f461aa40c28e83b2182de392a5/users/1352337746814a5d96149db8c7b9e81b/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:54.214612 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091519e-516d-4678-8f59-563834651766 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd0c79f461aa40c28e83b2182de392a5, user_id=1352337746814a5d96149db8c7b9e81b, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:54.217389 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091519e-516d-4678-8f59-563834651766 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:54.235641 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1309/2621] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:06:54 2022] PUT /identity/v3/projects/dd0c79f461aa40c28e83b2182de392a5/users/1352337746814a5d96149db8c7b9e81b/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:06:54.241911 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7065b2e-580e-4fec-9060-b17331e254ee admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:54.256562 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7065b2e-580e-4fec-9060-b17331e254ee admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:54.281235 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:06:54.321990 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e692f5bd-fc46-4055-a86f-a5e7802d7ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:54.322631 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692f5bd-fc46-4055-a86f-a5e7802d7ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:54.322911 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692f5bd-fc46-4055-a86f-a5e7802d7ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:54.323183 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692f5bd-fc46-4055-a86f-a5e7802d7ae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:54.324117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e692f5bd-fc46-4055-a86f-a5e7802d7ae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:54.325745 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e692f5bd-fc46-4055-a86f-a5e7802d7ae5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:54.336112 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1313/2622] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:06:54.340469 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091519e-516d-4678-8f59-563834651766 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:54.346063 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:06:54.351767 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091519e-516d-4678-8f59-563834651766 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:54.357621 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4b611-e8dd-477c-88bb-9153621595b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:54.358315 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4b611-e8dd-477c-88bb-9153621595b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:54.358581 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4b611-e8dd-477c-88bb-9153621595b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:54.358862 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4b611-e8dd-477c-88bb-9153621595b2 admin admin] PATH_INFO: `/v3/projects/dd0c79f461aa40c28e83b2182de392a5/users/1352337746814a5d96149db8c7b9e81b/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:54.361090 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc4b611-e8dd-477c-88bb-9153621595b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd0c79f461aa40c28e83b2182de392a5, user_id=1352337746814a5d96149db8c7b9e81b, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:54.363944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc4b611-e8dd-477c-88bb-9153621595b2 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:54.375219 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1310/2623] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:06:54 2022] PUT /identity/v3/projects/dd0c79f461aa40c28e83b2182de392a5/users/1352337746814a5d96149db8c7b9e81b/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:06:54.385656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7f928-14f5-4eff-ae84-942c3d0ecb62 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:54.385983 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7f928-14f5-4eff-ae84-942c3d0ecb62 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:54.386259 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7f928-14f5-4eff-ae84-942c3d0ecb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:54.418554 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-19f7f928-14f5-4eff-ae84-942c3d0ecb62 None None] MFA Rules not processed for user `1352337746814a5d96149db8c7b9e81b`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:06:54.419843 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-19f7f928-14f5-4eff-ae84-942c3d0ecb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:54.489949 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1314/2624] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:06:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:06:54.505349 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4b611-e8dd-477c-88bb-9153621595b2 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:54.507541 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-ccc4b611-e8dd-477c-88bb-9153621595b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:54.574198 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4b611-e8dd-477c-88bb-9153621595b2 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:54.578548 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2b8ab-5c49-4ce7-b309-4889e6c30eed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:54.579175 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2b8ab-5c49-4ce7-b309-4889e6c30eed service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:54.579460 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2b8ab-5c49-4ce7-b309-4889e6c30eed service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:54.579739 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2b8ab-5c49-4ce7-b309-4889e6c30eed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:54.591043 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2b8ab-5c49-4ce7-b309-4889e6c30eed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:54.593360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2b8ab-5c49-4ce7-b309-4889e6c30eed service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:54.605021 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1311/2625] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:06:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:06:54.730909 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7f928-14f5-4eff-ae84-942c3d0ecb62 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:54.742006 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7f928-14f5-4eff-ae84-942c3d0ecb62 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:54.746460 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56012e-f1fa-43e9-a7cd-4d015a6591a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:54.747085 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56012e-f1fa-43e9-a7cd-4d015a6591a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:54.747352 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56012e-f1fa-43e9-a7cd-4d015a6591a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:54.747648 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56012e-f1fa-43e9-a7cd-4d015a6591a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:54.819116 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee56012e-f1fa-43e9-a7cd-4d015a6591a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:54.820608 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee56012e-f1fa-43e9-a7cd-4d015a6591a0 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:54.829765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1315/2626] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:06:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:06:57.786817 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ac2de-d564-4189-9931-d5432b0b781f None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:57.787317 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ac2de-d564-4189-9931-d5432b0b781f None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:57.787542 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ac2de-d564-4189-9931-d5432b0b781f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:57.808404 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-aa7ac2de-d564-4189-9931-d5432b0b781f None None] MFA Rules not processed for user `3a8f3eb811ae40ad9cb64261f95a992e`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:06:57.809638 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-aa7ac2de-d564-4189-9931-d5432b0b781f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:57.870458 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1312/2627] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jul 26 05:06:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:06:57.893808 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56012e-f1fa-43e9-a7cd-4d015a6591a0 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:57.895371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ee56012e-f1fa-43e9-a7cd-4d015a6591a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:57.945395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:06:57.976656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56012e-f1fa-43e9-a7cd-4d015a6591a0 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:57.982332 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc398781-34d3-4929-997d-751c11694b30 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5218319aaa5424882714bffcbeee730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:57.982966 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc398781-34d3-4929-997d-751c11694b30 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:57.983243 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc398781-34d3-4929-997d-751c11694b30 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:57.983494 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc398781-34d3-4929-997d-751c11694b30 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:57.993969 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc398781-34d3-4929-997d-751c11694b30 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:57.995600 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc398781-34d3-4929-997d-751c11694b30 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:58.005359 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1316/2628] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jul 26 05:06:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 05:06:58.512455 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ac2de-d564-4189-9931-d5432b0b781f None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:58.581335 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ac2de-d564-4189-9931-d5432b0b781f None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:58.587569 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f31883-44cc-40ed-ad42-c924498a3015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:58.588474 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f31883-44cc-40ed-ad42-c924498a3015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:58.588991 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f31883-44cc-40ed-ad42-c924498a3015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:58.589408 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f31883-44cc-40ed-ad42-c924498a3015 admin admin] PATH_INFO: `/v3/users/cb73e14e2f0f46e085b24936f94be45d` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:58.590930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f31883-44cc-40ed-ad42-c924498a3015 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb73e14e2f0f46e085b24936f94be45d)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:58.593001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f31883-44cc-40ed-ad42-c924498a3015 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:58.650071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:06:58.661313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-50f31883-44cc-40ed-ad42-c924498a3015 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb73e14e2f0f46e085b24936f94be45d'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:58.672029 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-50f31883-44cc-40ed-ad42-c924498a3015 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb73e14e2f0f46e085b24936f94be45d'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:58.681443 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-50f31883-44cc-40ed-ad42-c924498a3015 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb73e14e2f0f46e085b24936f94be45d'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:58.682226 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-50f31883-44cc-40ed-ad42-c924498a3015 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb73e14e2f0f46e085b24936f94be45d'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:58.682900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-50f31883-44cc-40ed-ad42-c924498a3015 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb73e14e2f0f46e085b24936f94be45d'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:58.704736 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1313/2629] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:06:58 2022] DELETE /identity/v3/users/cb73e14e2f0f46e085b24936f94be45d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:06:58.709623 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92e9b4-1da0-47f5-a54f-93c54f13fd9a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:58.709925 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92e9b4-1da0-47f5-a54f-93c54f13fd9a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:58.710271 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92e9b4-1da0-47f5-a54f-93c54f13fd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:58.732067 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ac92e9b4-1da0-47f5-a54f-93c54f13fd9a None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:06:58.733640 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ac92e9b4-1da0-47f5-a54f-93c54f13fd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:58.805955 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1317/2630] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:06:58.819669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f31883-44cc-40ed-ad42-c924498a3015 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:58.821372 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-50f31883-44cc-40ed-ad42-c924498a3015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:58.897173 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f31883-44cc-40ed-ad42-c924498a3015 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:58.904401 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10168d1d-3dca-4a96-922f-b0a4d3600530 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:58.905118 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10168d1d-3dca-4a96-922f-b0a4d3600530 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:58.905407 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10168d1d-3dca-4a96-922f-b0a4d3600530 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:58.905683 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10168d1d-3dca-4a96-922f-b0a4d3600530 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:58.919187 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10168d1d-3dca-4a96-922f-b0a4d3600530 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:58.921569 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10168d1d-3dca-4a96-922f-b0a4d3600530 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:58.939748 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1314/2631] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:06:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:06:59.285182 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92e9b4-1da0-47f5-a54f-93c54f13fd9a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:59.286880 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ac92e9b4-1da0-47f5-a54f-93c54f13fd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:06:59.312771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92e9b4-1da0-47f5-a54f-93c54f13fd9a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:59.319683 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca9746-2df0-4692-8987-20083653e029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:59.320709 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca9746-2df0-4692-8987-20083653e029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:59.321276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca9746-2df0-4692-8987-20083653e029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:59.321622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca9746-2df0-4692-8987-20083653e029 admin admin] PATH_INFO: `/v3/projects/6d2a52aa395a41c1b3ab7eb4b2ba7f54` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:59.323374 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ca9746-2df0-4692-8987-20083653e029 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d2a52aa395a41c1b3ab7eb4b2ba7f54)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:59.325734 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ca9746-2df0-4692-8987-20083653e029 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:59.427790 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-74ca9746-2df0-4692-8987-20083653e029 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d2a52aa395a41c1b3ab7eb4b2ba7f54', 'request_id': 'req-74ca9746-2df0-4692-8987-20083653e029'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:59.441193 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-74ca9746-2df0-4692-8987-20083653e029 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d2a52aa395a41c1b3ab7eb4b2ba7f54', 'request_id': 'req-74ca9746-2df0-4692-8987-20083653e029'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:59.452848 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-74ca9746-2df0-4692-8987-20083653e029 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d2a52aa395a41c1b3ab7eb4b2ba7f54', 'request_id': 'req-74ca9746-2df0-4692-8987-20083653e029'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:59.468470 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-74ca9746-2df0-4692-8987-20083653e029 admin admin] The token cache is being invalidate because project 6d2a52aa395a41c1b3ab7eb4b2ba7f54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:06:59.469071 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-74ca9746-2df0-4692-8987-20083653e029 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:59.469898 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-74ca9746-2df0-4692-8987-20083653e029 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:59.472276 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1318/2632] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:06:59 2022] DELETE /identity/v3/projects/6d2a52aa395a41c1b3ab7eb4b2ba7f54 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:06:59.477090 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10168d1d-3dca-4a96-922f-b0a4d3600530 service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:06:59.491258 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10168d1d-3dca-4a96-922f-b0a4d3600530 service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:06:59.585023 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43daaa1e-bbe3-4e54-952a-558b509f36fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:06:59.585768 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43daaa1e-bbe3-4e54-952a-558b509f36fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:06:59.586159 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43daaa1e-bbe3-4e54-952a-558b509f36fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:06:59.586503 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43daaa1e-bbe3-4e54-952a-558b509f36fc admin admin] PATH_INFO: `/v3/users/1352337746814a5d96149db8c7b9e81b` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:06:59.587966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43daaa1e-bbe3-4e54-952a-558b509f36fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1352337746814a5d96149db8c7b9e81b)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:06:59.590252 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43daaa1e-bbe3-4e54-952a-558b509f36fc admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:06:59.661570 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-43daaa1e-bbe3-4e54-952a-558b509f36fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1352337746814a5d96149db8c7b9e81b'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:59.669195 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-43daaa1e-bbe3-4e54-952a-558b509f36fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1352337746814a5d96149db8c7b9e81b'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:59.673319 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:06:59.681139 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-43daaa1e-bbe3-4e54-952a-558b509f36fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1352337746814a5d96149db8c7b9e81b'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:59.682004 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-43daaa1e-bbe3-4e54-952a-558b509f36fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1352337746814a5d96149db8c7b9e81b'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:59.682929 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-43daaa1e-bbe3-4e54-952a-558b509f36fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1352337746814a5d96149db8c7b9e81b'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:06:59.709433 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1315/2633] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:06:59 2022] DELETE /identity/v3/users/1352337746814a5d96149db8c7b9e81b => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:00.048896 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca9746-2df0-4692-8987-20083653e029 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:00.050813 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-74ca9746-2df0-4692-8987-20083653e029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:00.070424 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca9746-2df0-4692-8987-20083653e029 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:00.145388 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:00.156011 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5488c4b-2d83-4b04-b33c-508f5153f30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:00.156770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5488c4b-2d83-4b04-b33c-508f5153f30c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:00.157467 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5488c4b-2d83-4b04-b33c-508f5153f30c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:00.157944 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5488c4b-2d83-4b04-b33c-508f5153f30c admin admin] PATH_INFO: `/v3/projects/dd0c79f461aa40c28e83b2182de392a5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:00.159669 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5488c4b-2d83-4b04-b33c-508f5153f30c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd0c79f461aa40c28e83b2182de392a5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:00.162055 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5488c4b-2d83-4b04-b33c-508f5153f30c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:00.220863 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e5488c4b-2d83-4b04-b33c-508f5153f30c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd0c79f461aa40c28e83b2182de392a5', 'request_id': 'req-e5488c4b-2d83-4b04-b33c-508f5153f30c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:00.230271 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e5488c4b-2d83-4b04-b33c-508f5153f30c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd0c79f461aa40c28e83b2182de392a5', 'request_id': 'req-e5488c4b-2d83-4b04-b33c-508f5153f30c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:00.242263 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e5488c4b-2d83-4b04-b33c-508f5153f30c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd0c79f461aa40c28e83b2182de392a5', 'request_id': 'req-e5488c4b-2d83-4b04-b33c-508f5153f30c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:00.257687 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e5488c4b-2d83-4b04-b33c-508f5153f30c admin admin] The token cache is being invalidate because project dd0c79f461aa40c28e83b2182de392a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:07:00.258374 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e5488c4b-2d83-4b04-b33c-508f5153f30c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:00.260467 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e5488c4b-2d83-4b04-b33c-508f5153f30c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:00.263155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1319/2634] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:07:00 2022] DELETE /identity/v3/projects/dd0c79f461aa40c28e83b2182de392a5 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:00.271298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75a590-26fc-4374-9b0f-7dbb615211f8 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:00.271858 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75a590-26fc-4374-9b0f-7dbb615211f8 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:00.272275 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75a590-26fc-4374-9b0f-7dbb615211f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:00.297695 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-ed75a590-26fc-4374-9b0f-7dbb615211f8 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:00.299201 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-ed75a590-26fc-4374-9b0f-7dbb615211f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:00.392425 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1316/2635] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:00.404117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3a2ee-da80-49d4-acc9-3c526796dab9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:00.404805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3a2ee-da80-49d4-acc9-3c526796dab9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:00.404805 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3a2ee-da80-49d4-acc9-3c526796dab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:00.430512 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6ee3a2ee-da80-49d4-acc9-3c526796dab9 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:00.431954 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6ee3a2ee-da80-49d4-acc9-3c526796dab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:00.441243 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1320/2636] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:00.446265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed75a590-26fc-4374-9b0f-7dbb615211f8 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:00.458406 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed75a590-26fc-4374-9b0f-7dbb615211f8 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:00.463849 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135d41f-2958-4baa-a5ca-e33f5592ec4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:00.464750 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135d41f-2958-4baa-a5ca-e33f5592ec4b admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:00.465486 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135d41f-2958-4baa-a5ca-e33f5592ec4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:00.465946 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135d41f-2958-4baa-a5ca-e33f5592ec4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:00.466747 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6135d41f-2958-4baa-a5ca-e33f5592ec4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:07:00.467061 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6135d41f-2958-4baa-a5ca-e33f5592ec4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:00.468826 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6135d41f-2958-4baa-a5ca-e33f5592ec4b admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:00.488111 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1317/2637] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:07:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:07:00.488591 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee3a2ee-da80-49d4-acc9-3c526796dab9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:00.499132 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee3a2ee-da80-49d4-acc9-3c526796dab9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:00.504474 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f92771-4140-4db1-b1c8-0b3ed1208791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:00.505227 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f92771-4140-4db1-b1c8-0b3ed1208791 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:00.505571 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f92771-4140-4db1-b1c8-0b3ed1208791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:00.505857 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f92771-4140-4db1-b1c8-0b3ed1208791 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:00.506556 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f92771-4140-4db1-b1c8-0b3ed1208791 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:00.508547 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f92771-4140-4db1-b1c8-0b3ed1208791 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:00.546122 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1321/2638] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:07:00 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:07:00.548417 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135d41f-2958-4baa-a5ca-e33f5592ec4b admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:00.560365 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135d41f-2958-4baa-a5ca-e33f5592ec4b admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:00.630263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f90c8-a850-4b9a-8718-ebe4e0bb5e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:00.630934 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f90c8-a850-4b9a-8718-ebe4e0bb5e7d admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:00.631222 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f90c8-a850-4b9a-8718-ebe4e0bb5e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:00.631502 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f90c8-a850-4b9a-8718-ebe4e0bb5e7d admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:00.632373 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f90c8-a850-4b9a-8718-ebe4e0bb5e7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:00.635102 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f90c8-a850-4b9a-8718-ebe4e0bb5e7d admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:00.661841 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1318/2639] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:07:00 2022] POST /identity/v3/users => generated 290 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:07:00.668871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f92771-4140-4db1-b1c8-0b3ed1208791 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:00.680077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f92771-4140-4db1-b1c8-0b3ed1208791 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:00.686249 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2510016a-c604-491f-aa51-65b44fd52906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:00.686875 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510016a-c604-491f-aa51-65b44fd52906 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:00.687151 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510016a-c604-491f-aa51-65b44fd52906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:00.687419 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510016a-c604-491f-aa51-65b44fd52906 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:00.688219 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2510016a-c604-491f-aa51-65b44fd52906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:00.689822 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2510016a-c604-491f-aa51-65b44fd52906 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:00.722277 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1322/2640] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:07:00.739957 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f90c8-a850-4b9a-8718-ebe4e0bb5e7d admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:00.750378 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f90c8-a850-4b9a-8718-ebe4e0bb5e7d admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:00.755107 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b285b48-82eb-4f44-bca9-4c5d2c1e3359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:00.755753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b285b48-82eb-4f44-bca9-4c5d2c1e3359 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:00.756031 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b285b48-82eb-4f44-bca9-4c5d2c1e3359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:00.756302 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b285b48-82eb-4f44-bca9-4c5d2c1e3359 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:00.757222 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b285b48-82eb-4f44-bca9-4c5d2c1e3359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:00.760125 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b285b48-82eb-4f44-bca9-4c5d2c1e3359 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:00.775814 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1319/2641] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:07:00.779746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2510016a-c604-491f-aa51-65b44fd52906 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:00.793735 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2510016a-c604-491f-aa51-65b44fd52906 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:00.798744 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a13ba-7566-4713-8077-4700069f868e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:00.799449 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a13ba-7566-4713-8077-4700069f868e admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:00.799449 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a13ba-7566-4713-8077-4700069f868e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:00.800033 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a13ba-7566-4713-8077-4700069f868e admin admin] PATH_INFO: `/v3/projects/007721dd02ad43e79cb6380dcf2a2dbd/users/00fef52c5807431d9b21c14afba73f02/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:00.817853 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63a13ba-7566-4713-8077-4700069f868e admin admin] RBAC: Authorizing `identity:create_grant(project_id=007721dd02ad43e79cb6380dcf2a2dbd, user_id=00fef52c5807431d9b21c14afba73f02, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:00.821226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63a13ba-7566-4713-8077-4700069f868e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:00.841787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1323/2642] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:07:00 2022] PUT /identity/v3/projects/007721dd02ad43e79cb6380dcf2a2dbd/users/00fef52c5807431d9b21c14afba73f02/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:00.853178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a9318-beb3-4845-a990-17507840aa6b None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:00.853470 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a9318-beb3-4845-a990-17507840aa6b None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:00.853752 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a9318-beb3-4845-a990-17507840aa6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:00.892722 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-e17a9318-beb3-4845-a990-17507840aa6b None None] MFA Rules not processed for user `00fef52c5807431d9b21c14afba73f02`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:00.894390 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-e17a9318-beb3-4845-a990-17507840aa6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:00.954014 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:00.997639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1320/2643] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:07:01.002788 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a13ba-7566-4713-8077-4700069f868e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:01.095328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a13ba-7566-4713-8077-4700069f868e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:01.102386 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793069f-3858-44fe-ad01-cfaa37155ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:01.103938 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793069f-3858-44fe-ad01-cfaa37155ed6 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:01.104370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793069f-3858-44fe-ad01-cfaa37155ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:01.104773 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793069f-3858-44fe-ad01-cfaa37155ed6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:01.105821 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7793069f-3858-44fe-ad01-cfaa37155ed6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:01.108239 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7793069f-3858-44fe-ad01-cfaa37155ed6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:01.140829 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1324/2644] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:07:01 2022] POST /identity/v3/projects => generated 377 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:07:01.145711 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a9318-beb3-4845-a990-17507840aa6b None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:01.169331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a9318-beb3-4845-a990-17507840aa6b None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:01.262163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60751f6d-be32-4cdc-adf6-6a0b3bb09ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:01.262819 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60751f6d-be32-4cdc-adf6-6a0b3bb09ef3 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:01.263112 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60751f6d-be32-4cdc-adf6-6a0b3bb09ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:01.263609 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60751f6d-be32-4cdc-adf6-6a0b3bb09ef3 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:01.264298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60751f6d-be32-4cdc-adf6-6a0b3bb09ef3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:01.266452 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60751f6d-be32-4cdc-adf6-6a0b3bb09ef3 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:01.290753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1321/2645] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:07:01 2022] POST /identity/v3/users => generated 289 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:07:01.296613 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793069f-3858-44fe-ad01-cfaa37155ed6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:01.309423 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793069f-3858-44fe-ad01-cfaa37155ed6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:01.315436 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff026f6f-84d1-4369-80af-c27b30d113a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:01.316207 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff026f6f-84d1-4369-80af-c27b30d113a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:01.316558 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff026f6f-84d1-4369-80af-c27b30d113a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:01.316908 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff026f6f-84d1-4369-80af-c27b30d113a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:01.317693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff026f6f-84d1-4369-80af-c27b30d113a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:01.321357 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff026f6f-84d1-4369-80af-c27b30d113a1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:01.326041 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:01.343120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60751f6d-be32-4cdc-adf6-6a0b3bb09ef3 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:01.344465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1325/2646] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:07:01.354046 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60751f6d-be32-4cdc-adf6-6a0b3bb09ef3 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:01.359763 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b92302-1820-4242-9697-9f85d026d8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:01.360025 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b92302-1820-4242-9697-9f85d026d8ce admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:01.360295 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b92302-1820-4242-9697-9f85d026d8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:01.360568 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b92302-1820-4242-9697-9f85d026d8ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:01.361282 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b92302-1820-4242-9697-9f85d026d8ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:01.362918 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b92302-1820-4242-9697-9f85d026d8ce admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:01.374192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1322/2647] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:07:01.378417 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff026f6f-84d1-4369-80af-c27b30d113a1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:01.400592 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff026f6f-84d1-4369-80af-c27b30d113a1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:01.405773 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727acbae-c55c-47e7-bdf6-bd5b82938e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:01.406550 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727acbae-c55c-47e7-bdf6-bd5b82938e09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:01.406907 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727acbae-c55c-47e7-bdf6-bd5b82938e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:01.407366 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727acbae-c55c-47e7-bdf6-bd5b82938e09 admin admin] PATH_INFO: `/v3/projects/eef3aca2935f471da315b629d45c5712/users/c3f40c24e8ac48acaf79e6c0a193ca6b/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:01.428869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727acbae-c55c-47e7-bdf6-bd5b82938e09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eef3aca2935f471da315b629d45c5712, user_id=c3f40c24e8ac48acaf79e6c0a193ca6b, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:01.432595 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727acbae-c55c-47e7-bdf6-bd5b82938e09 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:01.455586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1326/2648] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:07:01 2022] PUT /identity/v3/projects/eef3aca2935f471da315b629d45c5712/users/c3f40c24e8ac48acaf79e6c0a193ca6b/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:01.470611 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cea488-10f5-401c-8695-c42dfc53ed04 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:01.470907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cea488-10f5-401c-8695-c42dfc53ed04 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:01.471192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cea488-10f5-401c-8695-c42dfc53ed04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:01.512260 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-a5cea488-10f5-401c-8695-c42dfc53ed04 None None] MFA Rules not processed for user `c3f40c24e8ac48acaf79e6c0a193ca6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:01.513733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a5cea488-10f5-401c-8695-c42dfc53ed04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:01.610081 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1323/2649] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:01.614957 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727acbae-c55c-47e7-bdf6-bd5b82938e09 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:01.716529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727acbae-c55c-47e7-bdf6-bd5b82938e09 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:01.724901 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9215415-2f64-4b54-8c3e-d86931092259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:01.725853 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9215415-2f64-4b54-8c3e-d86931092259 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:01.726317 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9215415-2f64-4b54-8c3e-d86931092259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:01.726812 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9215415-2f64-4b54-8c3e-d86931092259 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:01.727651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9215415-2f64-4b54-8c3e-d86931092259 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:01.729836 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9215415-2f64-4b54-8c3e-d86931092259 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:01.772452 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1327/2650] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:07:01 2022] POST /identity/v3/projects => generated 379 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:07:01.774864 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cea488-10f5-401c-8695-c42dfc53ed04 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:01.791456 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cea488-10f5-401c-8695-c42dfc53ed04 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:01.870408 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:01.894823 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3300fa-eaa3-41ed-9aa5-ee82466fc603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:01.896022 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3300fa-eaa3-41ed-9aa5-ee82466fc603 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:01.896533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3300fa-eaa3-41ed-9aa5-ee82466fc603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:01.897226 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3300fa-eaa3-41ed-9aa5-ee82466fc603 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:01.898296 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3300fa-eaa3-41ed-9aa5-ee82466fc603 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:01.901475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3300fa-eaa3-41ed-9aa5-ee82466fc603 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:01.937654 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1324/2651] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:07:01 2022] POST /identity/v3/users => generated 296 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:07:01.938800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9215415-2f64-4b54-8c3e-d86931092259 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:01.951405 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9215415-2f64-4b54-8c3e-d86931092259 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:01.957534 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89190079-9742-4949-8c1c-e7bf0daa2db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:01.958290 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89190079-9742-4949-8c1c-e7bf0daa2db8 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:01.958628 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89190079-9742-4949-8c1c-e7bf0daa2db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:01.958968 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89190079-9742-4949-8c1c-e7bf0daa2db8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:01.959690 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89190079-9742-4949-8c1c-e7bf0daa2db8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:01.970058 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89190079-9742-4949-8c1c-e7bf0daa2db8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:01.997712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1328/2652] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:07:02.002579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3300fa-eaa3-41ed-9aa5-ee82466fc603 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:02.016905 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3300fa-eaa3-41ed-9aa5-ee82466fc603 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:02.029100 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40e34f-dd58-4ddb-acf6-f20661bcb4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:02.030636 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40e34f-dd58-4ddb-acf6-f20661bcb4b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:02.031124 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40e34f-dd58-4ddb-acf6-f20661bcb4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:02.031641 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40e34f-dd58-4ddb-acf6-f20661bcb4b7 admin admin] PATH_INFO: `/v3/projects/c08eb30ec9424c42a929063d1f5258e7/users/8eb10a7a6dec4f95ac3c267525d10bb3/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:02.053488 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40e34f-dd58-4ddb-acf6-f20661bcb4b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c08eb30ec9424c42a929063d1f5258e7, user_id=8eb10a7a6dec4f95ac3c267525d10bb3, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:02.077058 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40e34f-dd58-4ddb-acf6-f20661bcb4b7 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:02.107671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1325/2653] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:07:02 2022] PUT /identity/v3/projects/c08eb30ec9424c42a929063d1f5258e7/users/8eb10a7a6dec4f95ac3c267525d10bb3/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:02.111318 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89190079-9742-4949-8c1c-e7bf0daa2db8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:02.125522 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89190079-9742-4949-8c1c-e7bf0daa2db8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:02.245924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c93a6f-cd6a-4639-8b89-a22324ce5c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:02.246988 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c93a6f-cd6a-4639-8b89-a22324ce5c5c admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:02.247480 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c93a6f-cd6a-4639-8b89-a22324ce5c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:02.248019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c93a6f-cd6a-4639-8b89-a22324ce5c5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:02.248931 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c93a6f-cd6a-4639-8b89-a22324ce5c5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:02.250757 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c93a6f-cd6a-4639-8b89-a22324ce5c5c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:02.263393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1329/2654] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:07:02.268069 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40e34f-dd58-4ddb-acf6-f20661bcb4b7 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:02.291027 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40e34f-dd58-4ddb-acf6-f20661bcb4b7 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:02.299858 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7460eb-20d4-4b2b-b926-3d5a1284e038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:02.300626 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7460eb-20d4-4b2b-b926-3d5a1284e038 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:02.300900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7460eb-20d4-4b2b-b926-3d5a1284e038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:02.301176 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7460eb-20d4-4b2b-b926-3d5a1284e038 admin admin] PATH_INFO: `/v3/projects/c08eb30ec9424c42a929063d1f5258e7/users/8eb10a7a6dec4f95ac3c267525d10bb3/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:02.303629 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7460eb-20d4-4b2b-b926-3d5a1284e038 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c08eb30ec9424c42a929063d1f5258e7, user_id=8eb10a7a6dec4f95ac3c267525d10bb3, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:02.306631 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7460eb-20d4-4b2b-b926-3d5a1284e038 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:02.323351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1326/2655] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:07:02 2022] PUT /identity/v3/projects/c08eb30ec9424c42a929063d1f5258e7/users/8eb10a7a6dec4f95ac3c267525d10bb3/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:02.339727 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6fe1c-8b07-484a-b64f-e137d297ce8a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:02.340005 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6fe1c-8b07-484a-b64f-e137d297ce8a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:02.340292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6fe1c-8b07-484a-b64f-e137d297ce8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:02.413108 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4fc6fe1c-8b07-484a-b64f-e137d297ce8a None None] MFA Rules not processed for user `8eb10a7a6dec4f95ac3c267525d10bb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:02.415803 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4fc6fe1c-8b07-484a-b64f-e137d297ce8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:02.456384 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:02.499886 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1330/2656] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:02 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:02.512325 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7460eb-20d4-4b2b-b926-3d5a1284e038 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:02.514840 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-6a7460eb-20d4-4b2b-b926-3d5a1284e038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:02.656620 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7460eb-20d4-4b2b-b926-3d5a1284e038 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:02.664216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15ebee-3aa7-47ac-8e78-16e0f8c01673 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:02.665135 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15ebee-3aa7-47ac-8e78-16e0f8c01673 service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:02.665634 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15ebee-3aa7-47ac-8e78-16e0f8c01673 service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:02.666320 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15ebee-3aa7-47ac-8e78-16e0f8c01673 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:02.714060 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c15ebee-3aa7-47ac-8e78-16e0f8c01673 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:02.717183 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c15ebee-3aa7-47ac-8e78-16e0f8c01673 service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:02.736172 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1327/2657] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:07:02 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:07:02.817480 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6fe1c-8b07-484a-b64f-e137d297ce8a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:02.819607 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4fc6fe1c-8b07-484a-b64f-e137d297ce8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:02.834140 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6fe1c-8b07-484a-b64f-e137d297ce8a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:02.838103 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a9597-37f8-4cc4-93ec-f850bf128f42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:02.838959 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a9597-37f8-4cc4-93ec-f850bf128f42 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:02.839494 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a9597-37f8-4cc4-93ec-f850bf128f42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:02.839951 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a9597-37f8-4cc4-93ec-f850bf128f42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:02.956118 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a9597-37f8-4cc4-93ec-f850bf128f42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:02.957664 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a9597-37f8-4cc4-93ec-f850bf128f42 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:02.969513 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1331/2658] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:07:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:07:06.413700 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15ebee-3aa7-47ac-8e78-16e0f8c01673 service cinder] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:06.487411 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15ebee-3aa7-47ac-8e78-16e0f8c01673 service cinder] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:06.492298 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a3e93d-bf77-4a31-b1cd-781c5b380285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:06.493264 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3e93d-bf77-4a31-b1cd-781c5b380285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:06.493674 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3e93d-bf77-4a31-b1cd-781c5b380285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:06.493990 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3e93d-bf77-4a31-b1cd-781c5b380285 admin admin] PATH_INFO: `/v3/users/00fef52c5807431d9b21c14afba73f02` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:06.495338 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a3e93d-bf77-4a31-b1cd-781c5b380285 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00fef52c5807431d9b21c14afba73f02)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:06.497496 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a3e93d-bf77-4a31-b1cd-781c5b380285 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:06.546538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:06.556423 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-d9a3e93d-bf77-4a31-b1cd-781c5b380285 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00fef52c5807431d9b21c14afba73f02'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:06.567400 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-d9a3e93d-bf77-4a31-b1cd-781c5b380285 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00fef52c5807431d9b21c14afba73f02'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:06.578089 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-d9a3e93d-bf77-4a31-b1cd-781c5b380285 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00fef52c5807431d9b21c14afba73f02'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:06.578969 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-d9a3e93d-bf77-4a31-b1cd-781c5b380285 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00fef52c5807431d9b21c14afba73f02'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:06.579976 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-d9a3e93d-bf77-4a31-b1cd-781c5b380285 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00fef52c5807431d9b21c14afba73f02'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:06.613448 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1328/2659] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:07:06 2022] DELETE /identity/v3/users/00fef52c5807431d9b21c14afba73f02 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:06.615185 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36c566-4585-4c3e-ada5-74c7c7621422 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:06.615578 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36c566-4585-4c3e-ada5-74c7c7621422 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:06.615770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36c566-4585-4c3e-ada5-74c7c7621422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:06.649678 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fb36c566-4585-4c3e-ada5-74c7c7621422 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:06.655883 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fb36c566-4585-4c3e-ada5-74c7c7621422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:06.743923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1332/2660] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:07:06.753147 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a3e93d-bf77-4a31-b1cd-781c5b380285 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:06.754961 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-d9a3e93d-bf77-4a31-b1cd-781c5b380285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:06.835748 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a3e93d-bf77-4a31-b1cd-781c5b380285 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:06.842630 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139207b4-3e8e-4990-a33a-0b45a1349777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:06.843342 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139207b4-3e8e-4990-a33a-0b45a1349777 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:06.843685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139207b4-3e8e-4990-a33a-0b45a1349777 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:06.843959 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139207b4-3e8e-4990-a33a-0b45a1349777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:06.855969 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139207b4-3e8e-4990-a33a-0b45a1349777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:06.857546 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139207b4-3e8e-4990-a33a-0b45a1349777 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:06.871333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1329/2661] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:07:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:07:07.256240 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36c566-4585-4c3e-ada5-74c7c7621422 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:07.257852 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fb36c566-4585-4c3e-ada5-74c7c7621422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:07.271517 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36c566-4585-4c3e-ada5-74c7c7621422 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:07.281052 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d26896-bfbb-497a-880f-25e286ae6531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:07.281889 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d26896-bfbb-497a-880f-25e286ae6531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:07.282179 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d26896-bfbb-497a-880f-25e286ae6531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:07.282473 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d26896-bfbb-497a-880f-25e286ae6531 admin admin] PATH_INFO: `/v3/projects/007721dd02ad43e79cb6380dcf2a2dbd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:07.284076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d26896-bfbb-497a-880f-25e286ae6531 admin admin] RBAC: Authorizing `identity:delete_project(project_id=007721dd02ad43e79cb6380dcf2a2dbd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:07.286270 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d26896-bfbb-497a-880f-25e286ae6531 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:07.365006 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-78d26896-bfbb-497a-880f-25e286ae6531 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '007721dd02ad43e79cb6380dcf2a2dbd', 'request_id': 'req-78d26896-bfbb-497a-880f-25e286ae6531'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:07.372667 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-78d26896-bfbb-497a-880f-25e286ae6531 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '007721dd02ad43e79cb6380dcf2a2dbd', 'request_id': 'req-78d26896-bfbb-497a-880f-25e286ae6531'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:07.383819 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-78d26896-bfbb-497a-880f-25e286ae6531 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '007721dd02ad43e79cb6380dcf2a2dbd', 'request_id': 'req-78d26896-bfbb-497a-880f-25e286ae6531'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:07.387063 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:07.402428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-78d26896-bfbb-497a-880f-25e286ae6531 admin admin] The token cache is being invalidate because project 007721dd02ad43e79cb6380dcf2a2dbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:07:07.403330 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-78d26896-bfbb-497a-880f-25e286ae6531 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:07.405456 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-78d26896-bfbb-497a-880f-25e286ae6531 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:07.407453 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1333/2662] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:07:07 2022] DELETE /identity/v3/projects/007721dd02ad43e79cb6380dcf2a2dbd => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:07.411895 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139207b4-3e8e-4990-a33a-0b45a1349777 service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:07.423028 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139207b4-3e8e-4990-a33a-0b45a1349777 service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:07.498750 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3cf92-f71d-4f32-b6ba-104af435b52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:07.499463 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3cf92-f71d-4f32-b6ba-104af435b52d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:07.499802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3cf92-f71d-4f32-b6ba-104af435b52d admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:07.500087 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3cf92-f71d-4f32-b6ba-104af435b52d admin admin] PATH_INFO: `/v3/users/c3f40c24e8ac48acaf79e6c0a193ca6b` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:07.501400 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d3cf92-f71d-4f32-b6ba-104af435b52d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3f40c24e8ac48acaf79e6c0a193ca6b)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:07.503443 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d3cf92-f71d-4f32-b6ba-104af435b52d admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:07.564599 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-32d3cf92-f71d-4f32-b6ba-104af435b52d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3f40c24e8ac48acaf79e6c0a193ca6b'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:07.573900 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-32d3cf92-f71d-4f32-b6ba-104af435b52d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3f40c24e8ac48acaf79e6c0a193ca6b'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:07.579142 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:07.584479 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-32d3cf92-f71d-4f32-b6ba-104af435b52d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3f40c24e8ac48acaf79e6c0a193ca6b'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:07.585170 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-32d3cf92-f71d-4f32-b6ba-104af435b52d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3f40c24e8ac48acaf79e6c0a193ca6b'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:07.586681 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-32d3cf92-f71d-4f32-b6ba-104af435b52d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3f40c24e8ac48acaf79e6c0a193ca6b'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:07.609003 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1330/2663] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:07:07 2022] DELETE /identity/v3/users/c3f40c24e8ac48acaf79e6c0a193ca6b => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:07.973135 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d26896-bfbb-497a-880f-25e286ae6531 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:07.976784 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-78d26896-bfbb-497a-880f-25e286ae6531 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:07.999369 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d26896-bfbb-497a-880f-25e286ae6531 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:08.065613 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3cf92-f71d-4f32-b6ba-104af435b52d admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:08.067596 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-32d3cf92-f71d-4f32-b6ba-104af435b52d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:08.087764 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093703c-2a1d-43e4-bb1b-2bfbfa5a3e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:08.088727 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093703c-2a1d-43e4-bb1b-2bfbfa5a3e99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:08.089072 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093703c-2a1d-43e4-bb1b-2bfbfa5a3e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:08.089448 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093703c-2a1d-43e4-bb1b-2bfbfa5a3e99 admin admin] PATH_INFO: `/v3/projects/eef3aca2935f471da315b629d45c5712` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:08.091825 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093703c-2a1d-43e4-bb1b-2bfbfa5a3e99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eef3aca2935f471da315b629d45c5712)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:08.093870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093703c-2a1d-43e4-bb1b-2bfbfa5a3e99 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:08.170828 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3cf92-f71d-4f32-b6ba-104af435b52d admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:08.192728 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3093703c-2a1d-43e4-bb1b-2bfbfa5a3e99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eef3aca2935f471da315b629d45c5712', 'request_id': 'req-3093703c-2a1d-43e4-bb1b-2bfbfa5a3e99'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:08.202121 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3093703c-2a1d-43e4-bb1b-2bfbfa5a3e99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eef3aca2935f471da315b629d45c5712', 'request_id': 'req-3093703c-2a1d-43e4-bb1b-2bfbfa5a3e99'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:08.211441 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3093703c-2a1d-43e4-bb1b-2bfbfa5a3e99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eef3aca2935f471da315b629d45c5712', 'request_id': 'req-3093703c-2a1d-43e4-bb1b-2bfbfa5a3e99'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:08.236453 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016c49e-ed40-4de6-9f42-7e5ca9c179ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:08.237217 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016c49e-ed40-4de6-9f42-7e5ca9c179ad service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:08.237562 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016c49e-ed40-4de6-9f42-7e5ca9c179ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:08.237879 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016c49e-ed40-4de6-9f42-7e5ca9c179ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:08.239070 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3093703c-2a1d-43e4-bb1b-2bfbfa5a3e99 admin admin] The token cache is being invalidate because project eef3aca2935f471da315b629d45c5712 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:07:08.240075 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3093703c-2a1d-43e4-bb1b-2bfbfa5a3e99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:08.241887 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-3016c49e-ed40-4de6-9f42-7e5ca9c179ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:08.244126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3093703c-2a1d-43e4-bb1b-2bfbfa5a3e99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:08.246968 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1334/2664] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:07:07 2022] DELETE /identity/v3/projects/eef3aca2935f471da315b629d45c5712 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:07:08.250856 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093703c-2a1d-43e4-bb1b-2bfbfa5a3e99 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:08.253390 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3093703c-2a1d-43e4-bb1b-2bfbfa5a3e99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:08.341314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3016c49e-ed40-4de6-9f42-7e5ca9c179ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:08.342937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3016c49e-ed40-4de6-9f42-7e5ca9c179ad service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:08.345533 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093703c-2a1d-43e4-bb1b-2bfbfa5a3e99 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:08.353071 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e56c6f-8497-4f6f-a227-1c452bac501d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:08.354153 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e56c6f-8497-4f6f-a227-1c452bac501d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:08.354731 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e56c6f-8497-4f6f-a227-1c452bac501d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:08.355184 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e56c6f-8497-4f6f-a227-1c452bac501d admin admin] PATH_INFO: `/v3/users/8eb10a7a6dec4f95ac3c267525d10bb3` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:08.356396 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1331/2665] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:07:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:07:08.360553 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e56c6f-8497-4f6f-a227-1c452bac501d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eb10a7a6dec4f95ac3c267525d10bb3)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:08.362539 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e56c6f-8497-4f6f-a227-1c452bac501d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:08.403261 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:08.443925 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d0e56c6f-8497-4f6f-a227-1c452bac501d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eb10a7a6dec4f95ac3c267525d10bb3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:08.460048 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d0e56c6f-8497-4f6f-a227-1c452bac501d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eb10a7a6dec4f95ac3c267525d10bb3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:08.461394 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d0e56c6f-8497-4f6f-a227-1c452bac501d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eb10a7a6dec4f95ac3c267525d10bb3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:08.481402 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d0e56c6f-8497-4f6f-a227-1c452bac501d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eb10a7a6dec4f95ac3c267525d10bb3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:08.482550 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d0e56c6f-8497-4f6f-a227-1c452bac501d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eb10a7a6dec4f95ac3c267525d10bb3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:08.507865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1335/2666] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:07:08 2022] DELETE /identity/v3/users/8eb10a7a6dec4f95ac3c267525d10bb3 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:08.874059 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016c49e-ed40-4de6-9f42-7e5ca9c179ad service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:08.875861 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-3016c49e-ed40-4de6-9f42-7e5ca9c179ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:08.916565 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016c49e-ed40-4de6-9f42-7e5ca9c179ad service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:08.986007 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:09.006149 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0f235-63f7-499c-a2ae-67c7bc5c6f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:09.007356 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0f235-63f7-499c-a2ae-67c7bc5c6f86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:09.007950 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0f235-63f7-499c-a2ae-67c7bc5c6f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:09.008486 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0f235-63f7-499c-a2ae-67c7bc5c6f86 admin admin] PATH_INFO: `/v3/projects/c08eb30ec9424c42a929063d1f5258e7` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:09.010481 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0f235-63f7-499c-a2ae-67c7bc5c6f86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c08eb30ec9424c42a929063d1f5258e7)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:09.012955 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0f235-63f7-499c-a2ae-67c7bc5c6f86 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:09.092854 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-4fd0f235-63f7-499c-a2ae-67c7bc5c6f86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c08eb30ec9424c42a929063d1f5258e7', 'request_id': 'req-4fd0f235-63f7-499c-a2ae-67c7bc5c6f86'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:09.104605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-4fd0f235-63f7-499c-a2ae-67c7bc5c6f86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c08eb30ec9424c42a929063d1f5258e7', 'request_id': 'req-4fd0f235-63f7-499c-a2ae-67c7bc5c6f86'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:09.113646 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-4fd0f235-63f7-499c-a2ae-67c7bc5c6f86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c08eb30ec9424c42a929063d1f5258e7', 'request_id': 'req-4fd0f235-63f7-499c-a2ae-67c7bc5c6f86'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:09.126617 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-4fd0f235-63f7-499c-a2ae-67c7bc5c6f86 admin admin] The token cache is being invalidate because project c08eb30ec9424c42a929063d1f5258e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:07:09.126804 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-4fd0f235-63f7-499c-a2ae-67c7bc5c6f86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:09.127865 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-4fd0f235-63f7-499c-a2ae-67c7bc5c6f86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:09.131592 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1332/2667] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:07:08 2022] DELETE /identity/v3/projects/c08eb30ec9424c42a929063d1f5258e7 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:07:09.136565 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8cd97-fe84-4882-baae-74ec05111670 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:09.137124 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8cd97-fe84-4882-baae-74ec05111670 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:09.137124 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8cd97-fe84-4882-baae-74ec05111670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:09.160633 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d6a8cd97-fe84-4882-baae-74ec05111670 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:09.161866 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d6a8cd97-fe84-4882-baae-74ec05111670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:09.260447 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1336/2668] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:09.272772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06555f-2148-4aff-8c23-7cc2e1c09e0a None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:09.273714 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06555f-2148-4aff-8c23-7cc2e1c09e0a None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:09.274058 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06555f-2148-4aff-8c23-7cc2e1c09e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:09.299545 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-7a06555f-2148-4aff-8c23-7cc2e1c09e0a None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:09.300697 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-7a06555f-2148-4aff-8c23-7cc2e1c09e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:09.315602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1333/2669] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:09.319539 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8cd97-fe84-4882-baae-74ec05111670 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:09.345569 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8cd97-fe84-4882-baae-74ec05111670 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:09.350612 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987fbdf-da6c-43e9-935f-5aedb0dea401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:09.351762 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987fbdf-da6c-43e9-935f-5aedb0dea401 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:09.352208 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987fbdf-da6c-43e9-935f-5aedb0dea401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:09.352448 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987fbdf-da6c-43e9-935f-5aedb0dea401 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:09.353413 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f987fbdf-da6c-43e9-935f-5aedb0dea401 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:07:09.353856 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f987fbdf-da6c-43e9-935f-5aedb0dea401 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:09.355699 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f987fbdf-da6c-43e9-935f-5aedb0dea401 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:09.374746 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1337/2670] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:07:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:07:09.376997 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06555f-2148-4aff-8c23-7cc2e1c09e0a None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:09.393896 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06555f-2148-4aff-8c23-7cc2e1c09e0a None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:09.399153 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097acef-ee34-4973-8c08-a61e51447a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:09.400091 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097acef-ee34-4973-8c08-a61e51447a9a admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:09.400629 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097acef-ee34-4973-8c08-a61e51447a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:09.401079 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097acef-ee34-4973-8c08-a61e51447a9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:09.401945 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097acef-ee34-4973-8c08-a61e51447a9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:09.404097 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097acef-ee34-4973-8c08-a61e51447a9a admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:09.444616 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1334/2671] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:07:09 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:07:09.445494 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987fbdf-da6c-43e9-935f-5aedb0dea401 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:09.460582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987fbdf-da6c-43e9-935f-5aedb0dea401 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:09.533956 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89317966-fb3a-4b6c-a8f5-e86441f2f0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:09.534593 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89317966-fb3a-4b6c-a8f5-e86441f2f0a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:09.534883 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89317966-fb3a-4b6c-a8f5-e86441f2f0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:09.535166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89317966-fb3a-4b6c-a8f5-e86441f2f0a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:09.535879 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89317966-fb3a-4b6c-a8f5-e86441f2f0a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:09.537932 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89317966-fb3a-4b6c-a8f5-e86441f2f0a8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:09.570062 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1338/2672] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:07:09 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:07:09.571872 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097acef-ee34-4973-8c08-a61e51447a9a admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:09.585308 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097acef-ee34-4973-8c08-a61e51447a9a admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:09.590535 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a871ae0-f313-4710-a4e1-5687483144b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:09.591748 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a871ae0-f313-4710-a4e1-5687483144b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:09.592048 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a871ae0-f313-4710-a4e1-5687483144b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:09.592304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a871ae0-f313-4710-a4e1-5687483144b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:09.593064 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a871ae0-f313-4710-a4e1-5687483144b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:09.594707 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a871ae0-f313-4710-a4e1-5687483144b2 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:09.612994 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89317966-fb3a-4b6c-a8f5-e86441f2f0a8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:09.620024 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1335/2673] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:07:09.626721 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89317966-fb3a-4b6c-a8f5-e86441f2f0a8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:09.631871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94425b-cf06-4170-8e49-da6c4b395a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:09.632455 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94425b-cf06-4170-8e49-da6c4b395a04 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:09.632747 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94425b-cf06-4170-8e49-da6c4b395a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:09.633030 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94425b-cf06-4170-8e49-da6c4b395a04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:09.633744 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94425b-cf06-4170-8e49-da6c4b395a04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:09.635897 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94425b-cf06-4170-8e49-da6c4b395a04 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:09.652934 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1339/2674] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:07:09.654775 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a871ae0-f313-4710-a4e1-5687483144b2 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:09.668730 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a871ae0-f313-4710-a4e1-5687483144b2 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:09.673988 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfdb5b-aa2c-4277-b746-a18639dcc975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:09.675163 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfdb5b-aa2c-4277-b746-a18639dcc975 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:09.675683 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfdb5b-aa2c-4277-b746-a18639dcc975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:09.676189 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfdb5b-aa2c-4277-b746-a18639dcc975 admin admin] PATH_INFO: `/v3/projects/32108c1f825541b08254bacf45ceedea/users/98db651b5d2d4824b213803675cbdca2/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:09.695293 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dfdb5b-aa2c-4277-b746-a18639dcc975 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32108c1f825541b08254bacf45ceedea, user_id=98db651b5d2d4824b213803675cbdca2, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:09.698852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dfdb5b-aa2c-4277-b746-a18639dcc975 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:09.719145 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1336/2675] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:07:09 2022] PUT /identity/v3/projects/32108c1f825541b08254bacf45ceedea/users/98db651b5d2d4824b213803675cbdca2/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:09.720517 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9673b7e-2388-475c-900b-47bb5f5e6702 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:09.720817 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9673b7e-2388-475c-900b-47bb5f5e6702 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:09.721099 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9673b7e-2388-475c-900b-47bb5f5e6702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:09.738796 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:09.746586 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e9673b7e-2388-475c-900b-47bb5f5e6702 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:09.750103 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e9673b7e-2388-475c-900b-47bb5f5e6702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:09.828442 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1340/2676] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:07:09.836582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfdb5b-aa2c-4277-b746-a18639dcc975 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:09.840912 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-32dfdb5b-aa2c-4277-b746-a18639dcc975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:09.936966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfdb5b-aa2c-4277-b746-a18639dcc975 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:09.945234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8d655-87fe-4bcb-a26b-c82d82b3ddac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:09.946206 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8d655-87fe-4bcb-a26b-c82d82b3ddac service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:09.946582 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8d655-87fe-4bcb-a26b-c82d82b3ddac service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:09.947615 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8d655-87fe-4bcb-a26b-c82d82b3ddac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:09.961290 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be8d655-87fe-4bcb-a26b-c82d82b3ddac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:09.963170 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be8d655-87fe-4bcb-a26b-c82d82b3ddac service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:09.982374 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1337/2677] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:07:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:07:11.193490 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fce3f-c46e-4251-a179-9c8c29d7741e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:11.194008 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fce3f-c46e-4251-a179-9c8c29d7741e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:11.194388 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fce3f-c46e-4251-a179-9c8c29d7741e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:11.242471 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-960fce3f-c46e-4251-a179-9c8c29d7741e None None] MFA Rules not processed for user `98db651b5d2d4824b213803675cbdca2`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:11.245030 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-960fce3f-c46e-4251-a179-9c8c29d7741e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:11.430630 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1341/2678] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:11.435596 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8d655-87fe-4bcb-a26b-c82d82b3ddac service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:11.459889 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8d655-87fe-4bcb-a26b-c82d82b3ddac service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:11.466942 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef5882-3082-46d7-bf62-e5b513584a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:11.468160 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef5882-3082-46d7-bf62-e5b513584a22 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:11.468702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef5882-3082-46d7-bf62-e5b513584a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:11.469195 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef5882-3082-46d7-bf62-e5b513584a22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:11.470749 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef5882-3082-46d7-bf62-e5b513584a22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:11.473786 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef5882-3082-46d7-bf62-e5b513584a22 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:11.493384 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:11.530520 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1338/2679] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:07:11 2022] POST /identity/v3/projects => generated 371 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:07:11.534581 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960fce3f-c46e-4251-a179-9c8c29d7741e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:11.554583 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960fce3f-c46e-4251-a179-9c8c29d7741e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:11.645418 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce75c71-4841-4488-b467-97b6627432bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:11.645989 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce75c71-4841-4488-b467-97b6627432bf admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:11.646273 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce75c71-4841-4488-b467-97b6627432bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:11.646731 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce75c71-4841-4488-b467-97b6627432bf admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:11.647421 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce75c71-4841-4488-b467-97b6627432bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:11.649445 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce75c71-4841-4488-b467-97b6627432bf admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:11.703153 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1342/2680] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:07:11 2022] POST /identity/v3/users => generated 292 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:07:11.706965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef5882-3082-46d7-bf62-e5b513584a22 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:11.722563 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef5882-3082-46d7-bf62-e5b513584a22 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:11.727996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3c3a8-8178-4c74-9f80-538123517136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:11.728772 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3c3a8-8178-4c74-9f80-538123517136 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:11.729056 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3c3a8-8178-4c74-9f80-538123517136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:11.729327 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3c3a8-8178-4c74-9f80-538123517136 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:11.730045 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e3c3a8-8178-4c74-9f80-538123517136 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:11.731871 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e3c3a8-8178-4c74-9f80-538123517136 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:11.750238 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1339/2681] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:07:11.753953 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce75c71-4841-4488-b467-97b6627432bf admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:11.769751 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce75c71-4841-4488-b467-97b6627432bf admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:11.776468 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c88a95-bf50-4573-811d-044f0ae20b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:11.777748 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c88a95-bf50-4573-811d-044f0ae20b02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:11.778074 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c88a95-bf50-4573-811d-044f0ae20b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:11.778457 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c88a95-bf50-4573-811d-044f0ae20b02 admin admin] PATH_INFO: `/v3/projects/ab69f9fe40d14f01aa48e5ee41cb5b81/users/e246759867074d6e99dc8bdb13341910/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:11.807414 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c88a95-bf50-4573-811d-044f0ae20b02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab69f9fe40d14f01aa48e5ee41cb5b81, user_id=e246759867074d6e99dc8bdb13341910, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:11.815877 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c88a95-bf50-4573-811d-044f0ae20b02 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:11.850564 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1343/2682] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:07:11 2022] PUT /identity/v3/projects/ab69f9fe40d14f01aa48e5ee41cb5b81/users/e246759867074d6e99dc8bdb13341910/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:07:11.856057 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3c3a8-8178-4c74-9f80-538123517136 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:11.886827 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3c3a8-8178-4c74-9f80-538123517136 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:12.029038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c7f21-9e0d-4678-b5b3-4bdb4e10c56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:12.031577 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c7f21-9e0d-4678-b5b3-4bdb4e10c56a admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:12.032141 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c7f21-9e0d-4678-b5b3-4bdb4e10c56a admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:12.032648 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c7f21-9e0d-4678-b5b3-4bdb4e10c56a admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:12.033724 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c7f21-9e0d-4678-b5b3-4bdb4e10c56a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:12.036730 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c7f21-9e0d-4678-b5b3-4bdb4e10c56a admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:12.053767 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1340/2683] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:07:12.058044 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c88a95-bf50-4573-811d-044f0ae20b02 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:12.064152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:12.076875 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c88a95-bf50-4573-811d-044f0ae20b02 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:12.086658 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03617d60-8a67-4637-b290-a7feace9d14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:12.090226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03617d60-8a67-4637-b290-a7feace9d14a admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:12.090607 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03617d60-8a67-4637-b290-a7feace9d14a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:12.090887 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03617d60-8a67-4637-b290-a7feace9d14a admin admin] PATH_INFO: `/v3/projects/ab69f9fe40d14f01aa48e5ee41cb5b81/users/e246759867074d6e99dc8bdb13341910/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:12.096871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03617d60-8a67-4637-b290-a7feace9d14a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab69f9fe40d14f01aa48e5ee41cb5b81, user_id=e246759867074d6e99dc8bdb13341910, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:12.099817 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03617d60-8a67-4637-b290-a7feace9d14a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:12.112076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1344/2684] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:07:12 2022] PUT /identity/v3/projects/ab69f9fe40d14f01aa48e5ee41cb5b81/users/e246759867074d6e99dc8bdb13341910/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:13.458214 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c7f21-9e0d-4678-b5b3-4bdb4e10c56a admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:13.460093 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-2a7c7f21-9e0d-4678-b5b3-4bdb4e10c56a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:13.475448 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c7f21-9e0d-4678-b5b3-4bdb4e10c56a admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:13.548819 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5260fc-472c-4199-aa58-2550f6449fb8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:13.549660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5260fc-472c-4199-aa58-2550f6449fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:13.550138 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5260fc-472c-4199-aa58-2550f6449fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:13.558204 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eee2c0-7d08-47ae-914b-9d0c67393881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:13.558916 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eee2c0-7d08-47ae-914b-9d0c67393881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:13.559839 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eee2c0-7d08-47ae-914b-9d0c67393881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:13.560107 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eee2c0-7d08-47ae-914b-9d0c67393881 admin admin] PATH_INFO: `/v3/users/f996bed3f6064c6dbf04d26c7c04df7c` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:13.561577 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eee2c0-7d08-47ae-914b-9d0c67393881 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f996bed3f6064c6dbf04d26c7c04df7c)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:13.565798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eee2c0-7d08-47ae-914b-9d0c67393881 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:13.607623 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6e5260fc-472c-4199-aa58-2550f6449fb8 None None] MFA Rules not processed for user `e246759867074d6e99dc8bdb13341910`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:13.609812 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6e5260fc-472c-4199-aa58-2550f6449fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:13.632172 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-94eee2c0-7d08-47ae-914b-9d0c67393881 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f996bed3f6064c6dbf04d26c7c04df7c'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:13.642270 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-94eee2c0-7d08-47ae-914b-9d0c67393881 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f996bed3f6064c6dbf04d26c7c04df7c'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:13.657946 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-94eee2c0-7d08-47ae-914b-9d0c67393881 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f996bed3f6064c6dbf04d26c7c04df7c'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:13.659251 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-94eee2c0-7d08-47ae-914b-9d0c67393881 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f996bed3f6064c6dbf04d26c7c04df7c'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:13.660235 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-94eee2c0-7d08-47ae-914b-9d0c67393881 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f996bed3f6064c6dbf04d26c7c04df7c'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:13.675818 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:13.696046 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1341/2685] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:07:13 2022] DELETE /identity/v3/users/f996bed3f6064c6dbf04d26c7c04df7c => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:13.781119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1345/2686] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:13 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:13.794882 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eee2c0-7d08-47ae-914b-9d0c67393881 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:13.797917 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-94eee2c0-7d08-47ae-914b-9d0c67393881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:13.881261 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eee2c0-7d08-47ae-914b-9d0c67393881 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:13.885431 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256c9b3-be83-4070-8f88-18d33054c40c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:13.886129 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256c9b3-be83-4070-8f88-18d33054c40c service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:13.886421 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256c9b3-be83-4070-8f88-18d33054c40c service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:13.886699 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256c9b3-be83-4070-8f88-18d33054c40c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:13.888758 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-3256c9b3-be83-4070-8f88-18d33054c40c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:13.968524 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5260fc-472c-4199-aa58-2550f6449fb8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:13.971649 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6e5260fc-472c-4199-aa58-2550f6449fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:14.018759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256c9b3-be83-4070-8f88-18d33054c40c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:14.021197 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5260fc-472c-4199-aa58-2550f6449fb8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:14.025381 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256c9b3-be83-4070-8f88-18d33054c40c service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:14.036588 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1342/2687] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:07:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:07:14.099916 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94532556-13e0-41bb-8221-0af5b06b7e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:14.100646 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94532556-13e0-41bb-8221-0af5b06b7e75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:14.100940 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94532556-13e0-41bb-8221-0af5b06b7e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:14.101324 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94532556-13e0-41bb-8221-0af5b06b7e75 admin admin] PATH_INFO: `/v3/projects/27187d178a5f47b49035991ab566c2d9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:14.102838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94532556-13e0-41bb-8221-0af5b06b7e75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27187d178a5f47b49035991ab566c2d9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:14.105193 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94532556-13e0-41bb-8221-0af5b06b7e75 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:14.166150 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-94532556-13e0-41bb-8221-0af5b06b7e75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27187d178a5f47b49035991ab566c2d9', 'request_id': 'req-94532556-13e0-41bb-8221-0af5b06b7e75'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:14.175249 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-94532556-13e0-41bb-8221-0af5b06b7e75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27187d178a5f47b49035991ab566c2d9', 'request_id': 'req-94532556-13e0-41bb-8221-0af5b06b7e75'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:14.186283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-94532556-13e0-41bb-8221-0af5b06b7e75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27187d178a5f47b49035991ab566c2d9', 'request_id': 'req-94532556-13e0-41bb-8221-0af5b06b7e75'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:14.209081 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-94532556-13e0-41bb-8221-0af5b06b7e75 admin admin] The token cache is being invalidate because project 27187d178a5f47b49035991ab566c2d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:07:14.209656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-94532556-13e0-41bb-8221-0af5b06b7e75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:14.210384 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-94532556-13e0-41bb-8221-0af5b06b7e75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:14.213850 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1346/2688] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:07:13 2022] DELETE /identity/v3/projects/27187d178a5f47b49035991ab566c2d9 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:14.219208 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657926da-f8cf-4768-9acd-264897c97538 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:14.219491 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657926da-f8cf-4768-9acd-264897c97538 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:14.219798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657926da-f8cf-4768-9acd-264897c97538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:14.244600 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-657926da-f8cf-4768-9acd-264897c97538 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:14.245987 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-657926da-f8cf-4768-9acd-264897c97538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:14.321517 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1343/2689] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:14.332770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20d8fa-681e-4fb1-af89-55af25c3f310 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:14.333180 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20d8fa-681e-4fb1-af89-55af25c3f310 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:14.338337 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20d8fa-681e-4fb1-af89-55af25c3f310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:14.343876 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:14.356961 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3a20d8fa-681e-4fb1-af89-55af25c3f310 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:14.358820 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3a20d8fa-681e-4fb1-af89-55af25c3f310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:14.367511 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1347/2690] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:14.372845 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657926da-f8cf-4768-9acd-264897c97538 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:14.385260 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657926da-f8cf-4768-9acd-264897c97538 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:14.390722 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4405e95-dbd3-4b7a-8ee0-cb0a2f2666cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:14.391500 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4405e95-dbd3-4b7a-8ee0-cb0a2f2666cb admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:14.391803 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4405e95-dbd3-4b7a-8ee0-cb0a2f2666cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:14.392268 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4405e95-dbd3-4b7a-8ee0-cb0a2f2666cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:14.392773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4405e95-dbd3-4b7a-8ee0-cb0a2f2666cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:07:14.393443 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4405e95-dbd3-4b7a-8ee0-cb0a2f2666cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:14.394701 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4405e95-dbd3-4b7a-8ee0-cb0a2f2666cb admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:14.411720 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1344/2691] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:07:14.419457 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20d8fa-681e-4fb1-af89-55af25c3f310 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:14.455922 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20d8fa-681e-4fb1-af89-55af25c3f310 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:14.465236 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011dfb44-35cf-44d0-b09f-b675f69db1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:14.465983 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011dfb44-35cf-44d0-b09f-b675f69db1ed admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:14.466359 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011dfb44-35cf-44d0-b09f-b675f69db1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:14.466730 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011dfb44-35cf-44d0-b09f-b675f69db1ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:14.468150 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011dfb44-35cf-44d0-b09f-b675f69db1ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:14.470198 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011dfb44-35cf-44d0-b09f-b675f69db1ed admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:14.505676 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1348/2692] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:07:14 2022] POST /identity/v3/projects => generated 383 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:07:14.507380 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4405e95-dbd3-4b7a-8ee0-cb0a2f2666cb admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:14.518375 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4405e95-dbd3-4b7a-8ee0-cb0a2f2666cb admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:14.558786 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:14.589860 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de47e4-7e7f-46ee-b4f1-b190a62f8ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:14.590497 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de47e4-7e7f-46ee-b4f1-b190a62f8ebe admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:14.590782 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de47e4-7e7f-46ee-b4f1-b190a62f8ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:14.591063 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de47e4-7e7f-46ee-b4f1-b190a62f8ebe admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:14.591823 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de47e4-7e7f-46ee-b4f1-b190a62f8ebe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:14.593770 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de47e4-7e7f-46ee-b4f1-b190a62f8ebe admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:14.620343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1345/2693] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:07:14 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:07:14.622010 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011dfb44-35cf-44d0-b09f-b675f69db1ed admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:14.632935 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011dfb44-35cf-44d0-b09f-b675f69db1ed admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:14.637898 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ad822-f94d-4a88-8b1c-d8a219f3c453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:14.638608 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ad822-f94d-4a88-8b1c-d8a219f3c453 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:14.638896 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ad822-f94d-4a88-8b1c-d8a219f3c453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:14.639170 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ad822-f94d-4a88-8b1c-d8a219f3c453 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:14.639921 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ad822-f94d-4a88-8b1c-d8a219f3c453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:14.641535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ad822-f94d-4a88-8b1c-d8a219f3c453 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:14.652203 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1349/2694] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:07:14.656282 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de47e4-7e7f-46ee-b4f1-b190a62f8ebe admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:14.666619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de47e4-7e7f-46ee-b4f1-b190a62f8ebe admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:14.671317 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c4d5a7-4b97-43a7-8516-70614aafc963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:14.672049 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c4d5a7-4b97-43a7-8516-70614aafc963 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:14.672331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c4d5a7-4b97-43a7-8516-70614aafc963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:14.672610 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c4d5a7-4b97-43a7-8516-70614aafc963 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:14.673430 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c4d5a7-4b97-43a7-8516-70614aafc963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:14.675145 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c4d5a7-4b97-43a7-8516-70614aafc963 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:14.685554 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1346/2695] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:07:14.689856 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ad822-f94d-4a88-8b1c-d8a219f3c453 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:14.701206 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ad822-f94d-4a88-8b1c-d8a219f3c453 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:14.706469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff31e46-3ea6-4a44-a949-9567f98eff10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:14.707155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff31e46-3ea6-4a44-a949-9567f98eff10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:14.707440 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff31e46-3ea6-4a44-a949-9567f98eff10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:14.707754 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff31e46-3ea6-4a44-a949-9567f98eff10 admin admin] PATH_INFO: `/v3/projects/fd4481cb5d2f43c5b60db983132a3e16/users/92b9d212b86348c6b51a6b02818a0e80/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:14.725602 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff31e46-3ea6-4a44-a949-9567f98eff10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd4481cb5d2f43c5b60db983132a3e16, user_id=92b9d212b86348c6b51a6b02818a0e80, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:14.728432 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff31e46-3ea6-4a44-a949-9567f98eff10 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:14.745652 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1350/2696] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:07:14 2022] PUT /identity/v3/projects/fd4481cb5d2f43c5b60db983132a3e16/users/92b9d212b86348c6b51a6b02818a0e80/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:14.752169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819e4d8-7ace-43ad-a15d-3249eddaaba1 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:14.752459 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819e4d8-7ace-43ad-a15d-3249eddaaba1 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:14.752756 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819e4d8-7ace-43ad-a15d-3249eddaaba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:14.772989 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-0819e4d8-7ace-43ad-a15d-3249eddaaba1 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:14.774216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-0819e4d8-7ace-43ad-a15d-3249eddaaba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:14.849409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1347/2697] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:14.860598 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff31e46-3ea6-4a44-a949-9567f98eff10 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:14.863191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9ff31e46-3ea6-4a44-a949-9567f98eff10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:14.953703 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff31e46-3ea6-4a44-a949-9567f98eff10 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:14.959915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c93c5-29da-4479-9696-4b68fcba01e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:14.960685 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c93c5-29da-4479-9696-4b68fcba01e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:14.960985 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c93c5-29da-4479-9696-4b68fcba01e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:14.961520 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c93c5-29da-4479-9696-4b68fcba01e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:14.975986 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c93c5-29da-4479-9696-4b68fcba01e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:14.977748 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c93c5-29da-4479-9696-4b68fcba01e9 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:14.991789 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1351/2698] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:07:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:07:15.543592 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819e4d8-7ace-43ad-a15d-3249eddaaba1 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:15.546573 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-0819e4d8-7ace-43ad-a15d-3249eddaaba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:15.638121 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819e4d8-7ace-43ad-a15d-3249eddaaba1 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:15.645853 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f94d3-4665-415a-bdcf-28e7a2320859 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:15.647060 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f94d3-4665-415a-bdcf-28e7a2320859 service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:15.647679 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f94d3-4665-415a-bdcf-28e7a2320859 service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:15.648237 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f94d3-4665-415a-bdcf-28e7a2320859 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:15.650685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-0e9f94d3-4665-415a-bdcf-28e7a2320859 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:15.758960 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f94d3-4665-415a-bdcf-28e7a2320859 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:15.761485 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f94d3-4665-415a-bdcf-28e7a2320859 service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:15.775484 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1348/2699] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:07:15 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:07:17.734740 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db933a-51f8-438f-aab5-43a9de83aa19 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:17.735290 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db933a-51f8-438f-aab5-43a9de83aa19 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:17.735290 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db933a-51f8-438f-aab5-43a9de83aa19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:17.756616 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-37db933a-51f8-438f-aab5-43a9de83aa19 None None] MFA Rules not processed for user `3a8f3eb811ae40ad9cb64261f95a992e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:17.757849 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-37db933a-51f8-438f-aab5-43a9de83aa19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:17.825341 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1352/2700] 10.222.0.17 () {58 vars in 967 bytes} [Tue Jul 26 05:07:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:17.840309 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f94d3-4665-415a-bdcf-28e7a2320859 service cinder] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:17.842049 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-0e9f94d3-4665-415a-bdcf-28e7a2320859 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:17.857926 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:17.940888 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f94d3-4665-415a-bdcf-28e7a2320859 service cinder] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:17.947914 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22959ff9-daf4-427b-a8e0-57d23f94fc8b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5218319aaa5424882714bffcbeee730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:17.948839 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22959ff9-daf4-427b-a8e0-57d23f94fc8b service swift] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:17.949193 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22959ff9-daf4-427b-a8e0-57d23f94fc8b service swift] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:17.949429 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22959ff9-daf4-427b-a8e0-57d23f94fc8b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:17.964257 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22959ff9-daf4-427b-a8e0-57d23f94fc8b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:17.965975 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22959ff9-daf4-427b-a8e0-57d23f94fc8b service swift] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:17.986062 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1349/2701] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jul 26 05:07:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 05:07:19.557142 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5254e9-2bc8-4254-826c-a24e6f0d93ec None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:19.557578 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5254e9-2bc8-4254-826c-a24e6f0d93ec None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:19.557578 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5254e9-2bc8-4254-826c-a24e6f0d93ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:19.613474 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0b5254e9-2bc8-4254-826c-a24e6f0d93ec None None] MFA Rules not processed for user `92b9d212b86348c6b51a6b02818a0e80`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:19.614924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0b5254e9-2bc8-4254-826c-a24e6f0d93ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:19.665393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:19.699447 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1353/2702] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:19.715706 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22959ff9-daf4-427b-a8e0-57d23f94fc8b service swift] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:19.718483 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-22959ff9-daf4-427b-a8e0-57d23f94fc8b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:19.857618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22959ff9-daf4-427b-a8e0-57d23f94fc8b service swift] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:19.864196 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65df57-3730-4248-80c8-a5fbe7b028eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:19.865030 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65df57-3730-4248-80c8-a5fbe7b028eb service nova] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:19.865317 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65df57-3730-4248-80c8-a5fbe7b028eb service nova] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:19.865739 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65df57-3730-4248-80c8-a5fbe7b028eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:19.875045 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65df57-3730-4248-80c8-a5fbe7b028eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:19.876992 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65df57-3730-4248-80c8-a5fbe7b028eb service nova] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:19.887482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1350/2703] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:07:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:07:20.211713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5254e9-2bc8-4254-826c-a24e6f0d93ec None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:20.214408 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0b5254e9-2bc8-4254-826c-a24e6f0d93ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:20.234605 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5254e9-2bc8-4254-826c-a24e6f0d93ec None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:20.243529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3aba0-96f4-4d63-b4c2-b48eaacd8a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:20.244498 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3aba0-96f4-4d63-b4c2-b48eaacd8a29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:20.244795 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3aba0-96f4-4d63-b4c2-b48eaacd8a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:20.245068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3aba0-96f4-4d63-b4c2-b48eaacd8a29 admin admin] PATH_INFO: `/v3/users/ae6d25b14db34c6392999d0f7a929772` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:20.246433 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3aba0-96f4-4d63-b4c2-b48eaacd8a29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae6d25b14db34c6392999d0f7a929772)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:20.248446 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3aba0-96f4-4d63-b4c2-b48eaacd8a29 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:20.345478 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7cc3aba0-96f4-4d63-b4c2-b48eaacd8a29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae6d25b14db34c6392999d0f7a929772'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:20.362749 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7cc3aba0-96f4-4d63-b4c2-b48eaacd8a29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae6d25b14db34c6392999d0f7a929772'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:20.375596 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7cc3aba0-96f4-4d63-b4c2-b48eaacd8a29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae6d25b14db34c6392999d0f7a929772'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:20.386823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7cc3aba0-96f4-4d63-b4c2-b48eaacd8a29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae6d25b14db34c6392999d0f7a929772'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:20.393398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7cc3aba0-96f4-4d63-b4c2-b48eaacd8a29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae6d25b14db34c6392999d0f7a929772'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:20.420425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1354/2704] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:07:20 2022] DELETE /identity/v3/users/ae6d25b14db34c6392999d0f7a929772 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:20.644437 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65df57-3730-4248-80c8-a5fbe7b028eb service nova] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:20.656106 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65df57-3730-4248-80c8-a5fbe7b028eb service nova] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:20.732069 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de44f801-a842-4461-b449-52f5712bdb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:20.732948 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44f801-a842-4461-b449-52f5712bdb3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:20.733456 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44f801-a842-4461-b449-52f5712bdb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:20.735138 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44f801-a842-4461-b449-52f5712bdb3a admin admin] PATH_INFO: `/v3/projects/a30ecee1cac84583bb0e1c74eb333314` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:20.736751 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de44f801-a842-4461-b449-52f5712bdb3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a30ecee1cac84583bb0e1c74eb333314)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:20.738729 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de44f801-a842-4461-b449-52f5712bdb3a admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:20.773499 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:20.809150 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-de44f801-a842-4461-b449-52f5712bdb3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a30ecee1cac84583bb0e1c74eb333314', 'request_id': 'req-de44f801-a842-4461-b449-52f5712bdb3a'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:20.825666 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-de44f801-a842-4461-b449-52f5712bdb3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a30ecee1cac84583bb0e1c74eb333314', 'request_id': 'req-de44f801-a842-4461-b449-52f5712bdb3a'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:20.833231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-de44f801-a842-4461-b449-52f5712bdb3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a30ecee1cac84583bb0e1c74eb333314', 'request_id': 'req-de44f801-a842-4461-b449-52f5712bdb3a'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:20.848316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-de44f801-a842-4461-b449-52f5712bdb3a admin admin] The token cache is being invalidate because project a30ecee1cac84583bb0e1c74eb333314 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:07:20.848865 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-de44f801-a842-4461-b449-52f5712bdb3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:20.849955 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-de44f801-a842-4461-b449-52f5712bdb3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:20.851752 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1351/2705] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:07:20 2022] DELETE /identity/v3/projects/a30ecee1cac84583bb0e1c74eb333314 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:20.857145 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dab94-2b40-4fb9-b14c-8ed36b60d154 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:20.858152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dab94-2b40-4fb9-b14c-8ed36b60d154 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:20.858695 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dab94-2b40-4fb9-b14c-8ed36b60d154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:20.882102 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-740dab94-2b40-4fb9-b14c-8ed36b60d154 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:20.883817 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-740dab94-2b40-4fb9-b14c-8ed36b60d154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:20.950027 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1355/2706] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:07:20.958907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac9b14-9145-4c83-a14e-d4b73e4d200c None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:20.959753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac9b14-9145-4c83-a14e-d4b73e4d200c None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:20.960131 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac9b14-9145-4c83-a14e-d4b73e4d200c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:20.984809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-2eac9b14-9145-4c83-a14e-d4b73e4d200c None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:20.986051 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-2eac9b14-9145-4c83-a14e-d4b73e4d200c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:20.996291 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1352/2707] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:07:21.000703 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740dab94-2b40-4fb9-b14c-8ed36b60d154 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:21.011509 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740dab94-2b40-4fb9-b14c-8ed36b60d154 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:21.016506 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e011a3-c6a6-4338-b43f-fc5b448a545e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:21.017425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e011a3-c6a6-4338-b43f-fc5b448a545e admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:21.017716 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e011a3-c6a6-4338-b43f-fc5b448a545e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:21.017992 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e011a3-c6a6-4338-b43f-fc5b448a545e admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:21.018882 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e011a3-c6a6-4338-b43f-fc5b448a545e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:07:21.019237 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e011a3-c6a6-4338-b43f-fc5b448a545e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:21.020913 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e011a3-c6a6-4338-b43f-fc5b448a545e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:21.035038 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1356/2708] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:07:21.040111 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac9b14-9145-4c83-a14e-d4b73e4d200c None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:21.050645 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac9b14-9145-4c83-a14e-d4b73e4d200c None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:21.056693 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d53eaf-30a6-4a67-8551-32f3f5fdc1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:21.057286 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d53eaf-30a6-4a67-8551-32f3f5fdc1b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:21.057641 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d53eaf-30a6-4a67-8551-32f3f5fdc1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:21.058083 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d53eaf-30a6-4a67-8551-32f3f5fdc1b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:21.059211 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d53eaf-30a6-4a67-8551-32f3f5fdc1b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:21.061485 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d53eaf-30a6-4a67-8551-32f3f5fdc1b3 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:21.099162 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1353/2709] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:07:21 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:07:21.100588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e011a3-c6a6-4338-b43f-fc5b448a545e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:21.114948 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e011a3-c6a6-4338-b43f-fc5b448a545e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:21.156836 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:21.186471 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d0fe9-f1f3-4258-8438-6d0301e78297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:21.187114 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d0fe9-f1f3-4258-8438-6d0301e78297 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:21.187403 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d0fe9-f1f3-4258-8438-6d0301e78297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:21.187831 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d0fe9-f1f3-4258-8438-6d0301e78297 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:21.188513 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d0fe9-f1f3-4258-8438-6d0301e78297 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:21.190499 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d0fe9-f1f3-4258-8438-6d0301e78297 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:21.215533 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1357/2710] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:07:21 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:07:21.219967 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d53eaf-30a6-4a67-8551-32f3f5fdc1b3 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:21.232153 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d53eaf-30a6-4a67-8551-32f3f5fdc1b3 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:21.237626 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73814b9b-688e-4c0e-901c-1825443f7512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:21.238047 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73814b9b-688e-4c0e-901c-1825443f7512 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:21.238399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73814b9b-688e-4c0e-901c-1825443f7512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:21.238745 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73814b9b-688e-4c0e-901c-1825443f7512 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:21.239591 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73814b9b-688e-4c0e-901c-1825443f7512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:21.241483 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73814b9b-688e-4c0e-901c-1825443f7512 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:21.254305 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1354/2711] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:07:21.258982 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d0fe9-f1f3-4258-8438-6d0301e78297 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:21.269834 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d0fe9-f1f3-4258-8438-6d0301e78297 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:21.275468 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bc4de-cf54-48ca-ab20-601b94d5c732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:21.276202 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bc4de-cf54-48ca-ab20-601b94d5c732 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:21.276534 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bc4de-cf54-48ca-ab20-601b94d5c732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:21.276834 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bc4de-cf54-48ca-ab20-601b94d5c732 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:21.277520 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bc4de-cf54-48ca-ab20-601b94d5c732 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:21.279144 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bc4de-cf54-48ca-ab20-601b94d5c732 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:21.293556 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1358/2712] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:07:21.296652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73814b9b-688e-4c0e-901c-1825443f7512 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:21.310863 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73814b9b-688e-4c0e-901c-1825443f7512 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:21.317612 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf535e6-62f7-45cf-a52d-c7b2302a28c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:21.318306 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf535e6-62f7-45cf-a52d-c7b2302a28c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:21.318587 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf535e6-62f7-45cf-a52d-c7b2302a28c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:21.318864 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf535e6-62f7-45cf-a52d-c7b2302a28c8 admin admin] PATH_INFO: `/v3/projects/f1b92b49730c4ce7829e7f14b6870b43/users/c1e8fc3655db4d428bde11a8fc70c0dc/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:21.341069 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf535e6-62f7-45cf-a52d-c7b2302a28c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1b92b49730c4ce7829e7f14b6870b43, user_id=c1e8fc3655db4d428bde11a8fc70c0dc, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:21.344203 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf535e6-62f7-45cf-a52d-c7b2302a28c8 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:21.360212 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1355/2713] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:07:21 2022] PUT /identity/v3/projects/f1b92b49730c4ce7829e7f14b6870b43/users/c1e8fc3655db4d428bde11a8fc70c0dc/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:21.366571 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c07ce-de69-413b-acd2-20ea5237b095 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:21.366868 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c07ce-de69-413b-acd2-20ea5237b095 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:21.367150 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c07ce-de69-413b-acd2-20ea5237b095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:21.393747 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-095c07ce-de69-413b-acd2-20ea5237b095 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:21.395001 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-095c07ce-de69-413b-acd2-20ea5237b095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:21.490233 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1359/2714] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:21.500675 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf535e6-62f7-45cf-a52d-c7b2302a28c8 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:21.502458 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-7cf535e6-62f7-45cf-a52d-c7b2302a28c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:21.576914 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed8750-0162-4fa5-8f65-d5eeff35ca07 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:21.577480 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed8750-0162-4fa5-8f65-d5eeff35ca07 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:21.577765 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed8750-0162-4fa5-8f65-d5eeff35ca07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:21.607389 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0aed8750-0162-4fa5-8f65-d5eeff35ca07 None None] MFA Rules not processed for user `92b9d212b86348c6b51a6b02818a0e80`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:21.608964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0aed8750-0162-4fa5-8f65-d5eeff35ca07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:21.639489 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf535e6-62f7-45cf-a52d-c7b2302a28c8 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:21.648710 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8587e3-2d0d-4846-b807-d2b9097527fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:21.649349 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8587e3-2d0d-4846-b807-d2b9097527fc service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:21.649628 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8587e3-2d0d-4846-b807-d2b9097527fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:21.650748 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8587e3-2d0d-4846-b807-d2b9097527fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:21.670612 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8587e3-2d0d-4846-b807-d2b9097527fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:21.672443 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8587e3-2d0d-4846-b807-d2b9097527fc service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:21.698630 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1360/2715] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:07:21.703668 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1356/2716] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:07:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:07:27.490782 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0d423-7fc2-4975-9b8c-ae73188614f0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:27.491255 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0d423-7fc2-4975-9b8c-ae73188614f0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:27.491499 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0d423-7fc2-4975-9b8c-ae73188614f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:27.529830 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-31d0d423-7fc2-4975-9b8c-ae73188614f0 None None] MFA Rules not processed for user `c1e8fc3655db4d428bde11a8fc70c0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:27.531797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-31d0d423-7fc2-4975-9b8c-ae73188614f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:27.623869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:27.650082 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1361/2717] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:27.654864 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8587e3-2d0d-4846-b807-d2b9097527fc service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:27.665685 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8587e3-2d0d-4846-b807-d2b9097527fc service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:27.672006 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ce577-2298-427e-89cd-7dcb06e7864a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:27.673411 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ce577-2298-427e-89cd-7dcb06e7864a admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:27.674202 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ce577-2298-427e-89cd-7dcb06e7864a admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:27.674645 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ce577-2298-427e-89cd-7dcb06e7864a admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:27.675707 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768ce577-2298-427e-89cd-7dcb06e7864a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:27.678146 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768ce577-2298-427e-89cd-7dcb06e7864a admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:27.719016 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1357/2718] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:07:27 2022] POST /identity/v3/projects => generated 383 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:07:27.722887 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0d423-7fc2-4975-9b8c-ae73188614f0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:27.739673 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0d423-7fc2-4975-9b8c-ae73188614f0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:27.827424 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f66614-e79e-41b4-b1e8-2949c37febe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:27.828112 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f66614-e79e-41b4-b1e8-2949c37febe6 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:27.828570 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f66614-e79e-41b4-b1e8-2949c37febe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:27.828854 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f66614-e79e-41b4-b1e8-2949c37febe6 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:27.829679 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f66614-e79e-41b4-b1e8-2949c37febe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:27.831642 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f66614-e79e-41b4-b1e8-2949c37febe6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:27.879757 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1362/2719] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:07:27 2022] POST /identity/v3/users => generated 298 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:07:27.884131 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ce577-2298-427e-89cd-7dcb06e7864a admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:27.903123 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ce577-2298-427e-89cd-7dcb06e7864a admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:27.934370 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab932603-78ac-4c04-ba5b-cba3c075d255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:27.935313 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab932603-78ac-4c04-ba5b-cba3c075d255 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:27.935809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab932603-78ac-4c04-ba5b-cba3c075d255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:27.936792 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab932603-78ac-4c04-ba5b-cba3c075d255 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:27.938474 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab932603-78ac-4c04-ba5b-cba3c075d255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:27.940767 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab932603-78ac-4c04-ba5b-cba3c075d255 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:27.963609 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1358/2720] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:07:27.967073 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f66614-e79e-41b4-b1e8-2949c37febe6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:27.984154 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f66614-e79e-41b4-b1e8-2949c37febe6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:27.995990 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d04d5-8bf4-4b9a-8962-bdd799998f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:27.997331 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d04d5-8bf4-4b9a-8962-bdd799998f0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:27.997709 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d04d5-8bf4-4b9a-8962-bdd799998f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:27.998012 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d04d5-8bf4-4b9a-8962-bdd799998f0f admin admin] PATH_INFO: `/v3/projects/838457cc16d44a1f8a2b951f2d5e402a/users/434c5e94263e4f409ca4afdf093d3dde/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:28.026918 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d04d5-8bf4-4b9a-8962-bdd799998f0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=838457cc16d44a1f8a2b951f2d5e402a, user_id=434c5e94263e4f409ca4afdf093d3dde, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:28.029827 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d04d5-8bf4-4b9a-8962-bdd799998f0f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:28.045318 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1363/2721] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:07:27 2022] PUT /identity/v3/projects/838457cc16d44a1f8a2b951f2d5e402a/users/434c5e94263e4f409ca4afdf093d3dde/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:28.049359 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab932603-78ac-4c04-ba5b-cba3c075d255 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:28.060674 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab932603-78ac-4c04-ba5b-cba3c075d255 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:28.074192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:28.122681 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab386d-9389-480e-a9e6-a2bdf34bd95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:28.123405 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab386d-9389-480e-a9e6-a2bdf34bd95c admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:28.123794 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab386d-9389-480e-a9e6-a2bdf34bd95c admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:28.124141 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab386d-9389-480e-a9e6-a2bdf34bd95c admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:28.124925 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab386d-9389-480e-a9e6-a2bdf34bd95c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:28.126815 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab386d-9389-480e-a9e6-a2bdf34bd95c admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:28.138907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1359/2722] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:07:28.142812 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d04d5-8bf4-4b9a-8962-bdd799998f0f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:28.156859 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d04d5-8bf4-4b9a-8962-bdd799998f0f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:28.195080 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd466eb-d7e2-47b0-9b25-84fb7d67f2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:28.199901 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd466eb-d7e2-47b0-9b25-84fb7d67f2c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:28.200254 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd466eb-d7e2-47b0-9b25-84fb7d67f2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:28.200535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd466eb-d7e2-47b0-9b25-84fb7d67f2c6 admin admin] PATH_INFO: `/v3/projects/838457cc16d44a1f8a2b951f2d5e402a/users/434c5e94263e4f409ca4afdf093d3dde/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:28.202776 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd466eb-d7e2-47b0-9b25-84fb7d67f2c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=838457cc16d44a1f8a2b951f2d5e402a, user_id=434c5e94263e4f409ca4afdf093d3dde, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:28.205795 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd466eb-d7e2-47b0-9b25-84fb7d67f2c6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:28.225401 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1364/2723] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:07:28 2022] PUT /identity/v3/projects/838457cc16d44a1f8a2b951f2d5e402a/users/434c5e94263e4f409ca4afdf093d3dde/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:30.084222 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab386d-9389-480e-a9e6-a2bdf34bd95c admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:30.096218 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-28ab386d-9389-480e-a9e6-a2bdf34bd95c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:30.241722 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab386d-9389-480e-a9e6-a2bdf34bd95c admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:30.311510 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e1c55-290d-4cd0-8e93-eff25d0135f1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5049dba7ffde47b8af3e6fd55b5703e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:30.312415 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e1c55-290d-4cd0-8e93-eff25d0135f1 service placement] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:30.312821 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e1c55-290d-4cd0-8e93-eff25d0135f1 service placement] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:30.313196 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e1c55-290d-4cd0-8e93-eff25d0135f1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:30.315343 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-a23e1c55-290d-4cd0-8e93-eff25d0135f1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:30.336773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e1c55-290d-4cd0-8e93-eff25d0135f1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:30.339018 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e1c55-290d-4cd0-8e93-eff25d0135f1 service placement] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:30.356216 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1360/2724] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:07:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:07:30.917449 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd466eb-d7e2-47b0-9b25-84fb7d67f2c6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:30.920844 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bcd466eb-d7e2-47b0-9b25-84fb7d67f2c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:31.024585 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd466eb-d7e2-47b0-9b25-84fb7d67f2c6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:31.033351 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ecdec-864e-45e6-bc85-6bdde998845c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:31.033972 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ecdec-864e-45e6-bc85-6bdde998845c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:31.034243 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ecdec-864e-45e6-bc85-6bdde998845c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:31.034522 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ecdec-864e-45e6-bc85-6bdde998845c admin admin] PATH_INFO: `/v3/users/98db651b5d2d4824b213803675cbdca2` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:31.035783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784ecdec-864e-45e6-bc85-6bdde998845c admin admin] RBAC: Authorizing `identity:delete_user(user_id=98db651b5d2d4824b213803675cbdca2)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:31.037761 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784ecdec-864e-45e6-bc85-6bdde998845c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:31.104090 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:31.108187 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-784ecdec-864e-45e6-bc85-6bdde998845c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98db651b5d2d4824b213803675cbdca2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:31.121496 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-784ecdec-864e-45e6-bc85-6bdde998845c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98db651b5d2d4824b213803675cbdca2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:31.122855 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-784ecdec-864e-45e6-bc85-6bdde998845c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98db651b5d2d4824b213803675cbdca2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:31.140520 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-784ecdec-864e-45e6-bc85-6bdde998845c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98db651b5d2d4824b213803675cbdca2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:31.142242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-784ecdec-864e-45e6-bc85-6bdde998845c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98db651b5d2d4824b213803675cbdca2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:31.164230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1365/2725] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:07:30 2022] DELETE /identity/v3/users/98db651b5d2d4824b213803675cbdca2 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:31.443509 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e1c55-290d-4cd0-8e93-eff25d0135f1 service placement] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:31.466803 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e1c55-290d-4cd0-8e93-eff25d0135f1 service placement] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:31.598140 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:31.616465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6933f9a-a531-47e5-be23-86b7bdfc45ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:31.616465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6933f9a-a531-47e5-be23-86b7bdfc45ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:31.616465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6933f9a-a531-47e5-be23-86b7bdfc45ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:31.616465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6933f9a-a531-47e5-be23-86b7bdfc45ac admin admin] PATH_INFO: `/v3/projects/32108c1f825541b08254bacf45ceedea` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:31.616964 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6933f9a-a531-47e5-be23-86b7bdfc45ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=32108c1f825541b08254bacf45ceedea)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:31.620338 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6933f9a-a531-47e5-be23-86b7bdfc45ac admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:31.689810 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f6933f9a-a531-47e5-be23-86b7bdfc45ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32108c1f825541b08254bacf45ceedea', 'request_id': 'req-f6933f9a-a531-47e5-be23-86b7bdfc45ac'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:31.716178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f6933f9a-a531-47e5-be23-86b7bdfc45ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32108c1f825541b08254bacf45ceedea', 'request_id': 'req-f6933f9a-a531-47e5-be23-86b7bdfc45ac'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:31.751439 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f6933f9a-a531-47e5-be23-86b7bdfc45ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32108c1f825541b08254bacf45ceedea', 'request_id': 'req-f6933f9a-a531-47e5-be23-86b7bdfc45ac'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:31.801090 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f6933f9a-a531-47e5-be23-86b7bdfc45ac admin admin] The token cache is being invalidate because project 32108c1f825541b08254bacf45ceedea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:07:31.801301 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f6933f9a-a531-47e5-be23-86b7bdfc45ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:31.802516 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-f6933f9a-a531-47e5-be23-86b7bdfc45ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:31.809721 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1361/2726] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:07:31 2022] DELETE /identity/v3/projects/32108c1f825541b08254bacf45ceedea => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:31.811380 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ecdec-864e-45e6-bc85-6bdde998845c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:31.813504 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-784ecdec-864e-45e6-bc85-6bdde998845c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:31.831933 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ecdec-864e-45e6-bc85-6bdde998845c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:31.957790 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2be35-ffd9-468f-9904-1efb87a6cb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:31.959330 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2be35-ffd9-468f-9904-1efb87a6cb1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:31.959886 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2be35-ffd9-468f-9904-1efb87a6cb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:31.960348 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2be35-ffd9-468f-9904-1efb87a6cb1f admin admin] PATH_INFO: `/v3/users/e246759867074d6e99dc8bdb13341910` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:31.962298 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2be35-ffd9-468f-9904-1efb87a6cb1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e246759867074d6e99dc8bdb13341910)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:31.965611 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2be35-ffd9-468f-9904-1efb87a6cb1f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:32.047066 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1fd2be35-ffd9-468f-9904-1efb87a6cb1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e246759867074d6e99dc8bdb13341910'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:32.061153 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1fd2be35-ffd9-468f-9904-1efb87a6cb1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e246759867074d6e99dc8bdb13341910'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:32.061964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1fd2be35-ffd9-468f-9904-1efb87a6cb1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e246759867074d6e99dc8bdb13341910'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:32.074728 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1fd2be35-ffd9-468f-9904-1efb87a6cb1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e246759867074d6e99dc8bdb13341910'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:32.075715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1fd2be35-ffd9-468f-9904-1efb87a6cb1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e246759867074d6e99dc8bdb13341910'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:32.121766 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1366/2727] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:07:31 2022] DELETE /identity/v3/users/e246759867074d6e99dc8bdb13341910 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:32.368792 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6933f9a-a531-47e5-be23-86b7bdfc45ac admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:32.370703 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-f6933f9a-a531-47e5-be23-86b7bdfc45ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:32.407478 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6933f9a-a531-47e5-be23-86b7bdfc45ac admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:32.567888 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb988c0-4152-4169-a331-940d43832268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:32.569103 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb988c0-4152-4169-a331-940d43832268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:32.569534 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb988c0-4152-4169-a331-940d43832268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:32.570357 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb988c0-4152-4169-a331-940d43832268 admin admin] PATH_INFO: `/v3/projects/ab69f9fe40d14f01aa48e5ee41cb5b81` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:32.571801 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb988c0-4152-4169-a331-940d43832268 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab69f9fe40d14f01aa48e5ee41cb5b81)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:32.576103 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb988c0-4152-4169-a331-940d43832268 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:32.649734 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-adb988c0-4152-4169-a331-940d43832268 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab69f9fe40d14f01aa48e5ee41cb5b81', 'request_id': 'req-adb988c0-4152-4169-a331-940d43832268'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:32.661253 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-adb988c0-4152-4169-a331-940d43832268 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab69f9fe40d14f01aa48e5ee41cb5b81', 'request_id': 'req-adb988c0-4152-4169-a331-940d43832268'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:32.676542 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-adb988c0-4152-4169-a331-940d43832268 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab69f9fe40d14f01aa48e5ee41cb5b81', 'request_id': 'req-adb988c0-4152-4169-a331-940d43832268'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:32.689839 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-adb988c0-4152-4169-a331-940d43832268 admin admin] The token cache is being invalidate because project ab69f9fe40d14f01aa48e5ee41cb5b81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:07:32.690138 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-adb988c0-4152-4169-a331-940d43832268 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:32.691394 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-adb988c0-4152-4169-a331-940d43832268 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:32.693373 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1362/2728] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:07:32 2022] DELETE /identity/v3/projects/ab69f9fe40d14f01aa48e5ee41cb5b81 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:32.699892 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0161d-2ad5-489c-b0cf-3ec781769ab8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:32.700183 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0161d-2ad5-489c-b0cf-3ec781769ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:32.700464 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0161d-2ad5-489c-b0cf-3ec781769ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:32.727800 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fff0161d-2ad5-489c-b0cf-3ec781769ab8 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:32.729383 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fff0161d-2ad5-489c-b0cf-3ec781769ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:32.805842 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:32.843031 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1367/2729] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:32.943306 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5581f-d8ab-4f93-b437-0af3749d21aa None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:32.943306 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5581f-d8ab-4f93-b437-0af3749d21aa None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:32.943306 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5581f-d8ab-4f93-b437-0af3749d21aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:32.972991 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:32.992254 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-13c5581f-d8ab-4f93-b437-0af3749d21aa None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:32.993503 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-13c5581f-d8ab-4f93-b437-0af3749d21aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:33.003395 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1363/2730] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:33.007778 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0161d-2ad5-489c-b0cf-3ec781769ab8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:33.035362 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0161d-2ad5-489c-b0cf-3ec781769ab8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:33.043984 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739c72c-10a5-4a30-a809-6063ce40747c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:33.045174 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739c72c-10a5-4a30-a809-6063ce40747c admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:33.045659 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739c72c-10a5-4a30-a809-6063ce40747c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:33.046105 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739c72c-10a5-4a30-a809-6063ce40747c admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:33.046988 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739c72c-10a5-4a30-a809-6063ce40747c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:07:33.047454 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739c72c-10a5-4a30-a809-6063ce40747c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:33.049665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739c72c-10a5-4a30-a809-6063ce40747c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:33.084577 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1368/2731] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:07:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:07:33.088662 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5581f-d8ab-4f93-b437-0af3749d21aa None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:33.099033 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5581f-d8ab-4f93-b437-0af3749d21aa None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:33.104527 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad4b6bf-adca-49aa-9c99-a44acd7209f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:33.105184 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4b6bf-adca-49aa-9c99-a44acd7209f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:33.105467 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4b6bf-adca-49aa-9c99-a44acd7209f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:33.109663 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4b6bf-adca-49aa-9c99-a44acd7209f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:33.110896 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad4b6bf-adca-49aa-9c99-a44acd7209f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:33.112565 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad4b6bf-adca-49aa-9c99-a44acd7209f1 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:33.200986 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1364/2732] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:07:33 2022] POST /identity/v3/projects => generated 359 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:07:33.202366 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739c72c-10a5-4a30-a809-6063ce40747c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:33.217235 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739c72c-10a5-4a30-a809-6063ce40747c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:33.313439 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d972ad-9092-4b4c-a86c-7662ffada448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:33.314333 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d972ad-9092-4b4c-a86c-7662ffada448 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:33.314680 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d972ad-9092-4b4c-a86c-7662ffada448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:33.315186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d972ad-9092-4b4c-a86c-7662ffada448 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:33.316243 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d972ad-9092-4b4c-a86c-7662ffada448 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:33.319132 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d972ad-9092-4b4c-a86c-7662ffada448 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:33.354492 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1369/2733] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:07:33 2022] POST /identity/v3/users => generated 280 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:07:33.358286 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad4b6bf-adca-49aa-9c99-a44acd7209f1 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:33.377647 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad4b6bf-adca-49aa-9c99-a44acd7209f1 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:33.385279 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645fc2d5-2eab-4870-af3f-e0048bdb57af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:33.386342 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fc2d5-2eab-4870-af3f-e0048bdb57af admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:33.386836 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fc2d5-2eab-4870-af3f-e0048bdb57af admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:33.387321 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fc2d5-2eab-4870-af3f-e0048bdb57af admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:33.388385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645fc2d5-2eab-4870-af3f-e0048bdb57af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:33.394225 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645fc2d5-2eab-4870-af3f-e0048bdb57af admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:33.414617 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1365/2734] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:07:33.418623 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d972ad-9092-4b4c-a86c-7662ffada448 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:33.431398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d972ad-9092-4b4c-a86c-7662ffada448 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:33.438912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4082fb8-8dbb-47da-9752-29f60c7db06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:33.440031 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4082fb8-8dbb-47da-9752-29f60c7db06f admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:33.440528 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4082fb8-8dbb-47da-9752-29f60c7db06f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:33.441014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4082fb8-8dbb-47da-9752-29f60c7db06f admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:33.442660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4082fb8-8dbb-47da-9752-29f60c7db06f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:33.447331 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4082fb8-8dbb-47da-9752-29f60c7db06f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:33.471251 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1370/2735] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:07:33.476129 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645fc2d5-2eab-4870-af3f-e0048bdb57af admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:33.492297 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645fc2d5-2eab-4870-af3f-e0048bdb57af admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:33.504698 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d02a59-1816-4a95-8467-05a4053ba414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:33.505974 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d02a59-1816-4a95-8467-05a4053ba414 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:33.505974 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d02a59-1816-4a95-8467-05a4053ba414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:33.506311 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d02a59-1816-4a95-8467-05a4053ba414 admin admin] PATH_INFO: `/v3/projects/3563428ee90342cdae1a2e3b99500320/users/4b9c783d191945b383fc74e61f20e684/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:33.541150 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d02a59-1816-4a95-8467-05a4053ba414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3563428ee90342cdae1a2e3b99500320, user_id=4b9c783d191945b383fc74e61f20e684, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:33.544257 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d02a59-1816-4a95-8467-05a4053ba414 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:33.577732 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1366/2736] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:07:33 2022] PUT /identity/v3/projects/3563428ee90342cdae1a2e3b99500320/users/4b9c783d191945b383fc74e61f20e684/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:33.590165 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb5d7f-ff8c-4ea2-9a4a-906c7a82f7af None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:33.590165 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb5d7f-ff8c-4ea2-9a4a-906c7a82f7af None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:33.590165 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb5d7f-ff8c-4ea2-9a4a-906c7a82f7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:33.648338 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ebfb5d7f-ff8c-4ea2-9a4a-906c7a82f7af None None] MFA Rules not processed for user `4b9c783d191945b383fc74e61f20e684`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:33.649809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ebfb5d7f-ff8c-4ea2-9a4a-906c7a82f7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:33.698003 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044782b9-517e-4578-bd14-337e7b5f0127 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:33.699034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044782b9-517e-4578-bd14-337e7b5f0127 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:33.699467 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044782b9-517e-4578-bd14-337e7b5f0127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:33.764997 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-044782b9-517e-4578-bd14-337e7b5f0127 None None] MFA Rules not processed for user `434c5e94263e4f409ca4afdf093d3dde`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:33.767590 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-044782b9-517e-4578-bd14-337e7b5f0127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:33.812256 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1371/2737] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:33 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:33.854316 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb5d7f-ff8c-4ea2-9a4a-906c7a82f7af None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:33.862810 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ebfb5d7f-ff8c-4ea2-9a4a-906c7a82f7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:33.959290 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1367/2738] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:33.969566 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044782b9-517e-4578-bd14-337e7b5f0127 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:33.971254 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-044782b9-517e-4578-bd14-337e7b5f0127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:34.014142 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:34.058771 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb5d7f-ff8c-4ea2-9a4a-906c7a82f7af None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:34.067619 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe366b-674e-4113-9fc4-9d76398644e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:34.069380 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe366b-674e-4113-9fc4-9d76398644e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:34.069947 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe366b-674e-4113-9fc4-9d76398644e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:34.070375 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe366b-674e-4113-9fc4-9d76398644e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:34.091885 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbe366b-674e-4113-9fc4-9d76398644e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:34.093912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbe366b-674e-4113-9fc4-9d76398644e6 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:34.123687 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044782b9-517e-4578-bd14-337e7b5f0127 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:34.128346 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98367fd-52f0-44b4-bc15-5aa7095ff59c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6946a788d2ef49ff873ca8e81990527e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:34.129536 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98367fd-52f0-44b4-bc15-5aa7095ff59c service glance] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:34.130054 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98367fd-52f0-44b4-bc15-5aa7095ff59c service glance] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:34.130545 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98367fd-52f0-44b4-bc15-5aa7095ff59c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:34.133391 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-f98367fd-52f0-44b4-bc15-5aa7095ff59c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:34.135593 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1372/2739] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:07:33 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:07:34.184581 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe366b-674e-4113-9fc4-9d76398644e6 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:34.208302 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:34.258116 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98367fd-52f0-44b4-bc15-5aa7095ff59c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:34.263989 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98367fd-52f0-44b4-bc15-5aa7095ff59c service glance] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:34.283492 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe366b-674e-4113-9fc4-9d76398644e6 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:34.287744 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1368/2740] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:07:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:07:34.289099 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1412ae-0895-447f-90b5-582dc756035a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:34.289878 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1412ae-0895-447f-90b5-582dc756035a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:34.290261 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1412ae-0895-447f-90b5-582dc756035a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:34.290603 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1412ae-0895-447f-90b5-582dc756035a admin admin] PATH_INFO: `/v3/users/4b9c783d191945b383fc74e61f20e684` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:34.292886 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1412ae-0895-447f-90b5-582dc756035a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b9c783d191945b383fc74e61f20e684)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:34.294996 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1412ae-0895-447f-90b5-582dc756035a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:34.370332 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0a1412ae-0895-447f-90b5-582dc756035a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b9c783d191945b383fc74e61f20e684'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:34.394995 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0a1412ae-0895-447f-90b5-582dc756035a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b9c783d191945b383fc74e61f20e684'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:34.395277 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0a1412ae-0895-447f-90b5-582dc756035a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b9c783d191945b383fc74e61f20e684'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:34.406483 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0a1412ae-0895-447f-90b5-582dc756035a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b9c783d191945b383fc74e61f20e684'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:34.408234 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0a1412ae-0895-447f-90b5-582dc756035a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b9c783d191945b383fc74e61f20e684'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:34.436419 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1373/2741] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:07:34 2022] DELETE /identity/v3/users/4b9c783d191945b383fc74e61f20e684 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:34.438431 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88f660-bc75-4444-80a1-c617f25005fa None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:34.438724 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88f660-bc75-4444-80a1-c617f25005fa None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:34.439000 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88f660-bc75-4444-80a1-c617f25005fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:34.485117 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-4a88f660-bc75-4444-80a1-c617f25005fa None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:34.485698 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db0dc2-be47-4d4d-a500-b0a03802db86 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:34.486646 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-4a88f660-bc75-4444-80a1-c617f25005fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:34.487263 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db0dc2-be47-4d4d-a500-b0a03802db86 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:34.487662 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db0dc2-be47-4d4d-a500-b0a03802db86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:34.510785 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c4db0dc2-be47-4d4d-a500-b0a03802db86 None None] MFA Rules not processed for user `3a8f3eb811ae40ad9cb64261f95a992e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:34.512468 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c4db0dc2-be47-4d4d-a500-b0a03802db86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:34.568804 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1369/2742] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:34.580911 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88f660-bc75-4444-80a1-c617f25005fa None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:34.582606 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-4a88f660-bc75-4444-80a1-c617f25005fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:34.604155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1374/2743] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jul 26 05:07:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:34.621085 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db0dc2-be47-4d4d-a500-b0a03802db86 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:34.623603 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c4db0dc2-be47-4d4d-a500-b0a03802db86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:34.632609 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:34.672009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88f660-bc75-4444-80a1-c617f25005fa None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:34.676015 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b1dfc-3213-4a2d-a460-19a9124bc907 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:34.676826 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b1dfc-3213-4a2d-a460-19a9124bc907 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:34.677253 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b1dfc-3213-4a2d-a460-19a9124bc907 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:34.677539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b1dfc-3213-4a2d-a460-19a9124bc907 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:34.691055 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160b1dfc-3213-4a2d-a460-19a9124bc907 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:34.693482 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160b1dfc-3213-4a2d-a460-19a9124bc907 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:34.709641 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1370/2744] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:07:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:07:34.755983 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db0dc2-be47-4d4d-a500-b0a03802db86 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:34.766650 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940336e-a3e8-4dbe-a893-fb064e753ab1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5218319aaa5424882714bffcbeee730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:34.767509 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940336e-a3e8-4dbe-a893-fb064e753ab1 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:34.767917 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940336e-a3e8-4dbe-a893-fb064e753ab1 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:34.768602 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940336e-a3e8-4dbe-a893-fb064e753ab1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:34.778459 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940336e-a3e8-4dbe-a893-fb064e753ab1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:34.780716 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940336e-a3e8-4dbe-a893-fb064e753ab1 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:34.793308 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1375/2745] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jul 26 05:07:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 05:07:35.108669 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b1dfc-3213-4a2d-a460-19a9124bc907 service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:35.110232 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-160b1dfc-3213-4a2d-a460-19a9124bc907 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:35.142691 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b1dfc-3213-4a2d-a460-19a9124bc907 service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:35.165463 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba80756-c0ef-4809-82a9-85c201968ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:35.166536 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba80756-c0ef-4809-82a9-85c201968ebc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:35.166829 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba80756-c0ef-4809-82a9-85c201968ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:35.167521 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba80756-c0ef-4809-82a9-85c201968ebc admin admin] PATH_INFO: `/v3/projects/3563428ee90342cdae1a2e3b99500320` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:35.169416 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba80756-c0ef-4809-82a9-85c201968ebc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3563428ee90342cdae1a2e3b99500320)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:35.172250 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba80756-c0ef-4809-82a9-85c201968ebc admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:35.313580 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-7ba80756-c0ef-4809-82a9-85c201968ebc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3563428ee90342cdae1a2e3b99500320', 'request_id': 'req-7ba80756-c0ef-4809-82a9-85c201968ebc'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:35.329817 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-7ba80756-c0ef-4809-82a9-85c201968ebc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3563428ee90342cdae1a2e3b99500320', 'request_id': 'req-7ba80756-c0ef-4809-82a9-85c201968ebc'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:35.347042 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-7ba80756-c0ef-4809-82a9-85c201968ebc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3563428ee90342cdae1a2e3b99500320', 'request_id': 'req-7ba80756-c0ef-4809-82a9-85c201968ebc'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:35.362671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-7ba80756-c0ef-4809-82a9-85c201968ebc admin admin] The token cache is being invalidate because project 3563428ee90342cdae1a2e3b99500320 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:07:35.362965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-7ba80756-c0ef-4809-82a9-85c201968ebc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:35.364046 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-7ba80756-c0ef-4809-82a9-85c201968ebc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:35.366265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1371/2746] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:07:35 2022] DELETE /identity/v3/projects/3563428ee90342cdae1a2e3b99500320 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:35.371397 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6e28b-77d5-45de-be12-911f64a6dd23 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:35.372042 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6e28b-77d5-45de-be12-911f64a6dd23 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:35.372042 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6e28b-77d5-45de-be12-911f64a6dd23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:35.399119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c0d6e28b-77d5-45de-be12-911f64a6dd23 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:35.404180 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c0d6e28b-77d5-45de-be12-911f64a6dd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:35.501852 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1376/2747] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:35.512300 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5dc173-e0fc-43a7-8a94-799062d08e47 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:35.512596 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5dc173-e0fc-43a7-8a94-799062d08e47 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:35.512873 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5dc173-e0fc-43a7-8a94-799062d08e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:35.536578 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-fa5dc173-e0fc-43a7-8a94-799062d08e47 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:35.537818 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-fa5dc173-e0fc-43a7-8a94-799062d08e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:35.547588 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1372/2748] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:35.552016 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6e28b-77d5-45de-be12-911f64a6dd23 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:35.562870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6e28b-77d5-45de-be12-911f64a6dd23 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:35.568010 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d89935-7605-4c8a-945f-ca51c3ff6337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:35.568885 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d89935-7605-4c8a-945f-ca51c3ff6337 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:35.569168 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d89935-7605-4c8a-945f-ca51c3ff6337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:35.569440 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d89935-7605-4c8a-945f-ca51c3ff6337 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:35.570148 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d89935-7605-4c8a-945f-ca51c3ff6337 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:07:35.570451 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d89935-7605-4c8a-945f-ca51c3ff6337 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:35.572235 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d89935-7605-4c8a-945f-ca51c3ff6337 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:35.603243 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5dc173-e0fc-43a7-8a94-799062d08e47 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:35.608481 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:35.608683 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1377/2749] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:07:35.615615 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5dc173-e0fc-43a7-8a94-799062d08e47 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:35.631223 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e95e3-6d92-4980-8f26-4e85e1a0e841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:35.635387 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e95e3-6d92-4980-8f26-4e85e1a0e841 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:35.638868 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e95e3-6d92-4980-8f26-4e85e1a0e841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:35.639652 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e95e3-6d92-4980-8f26-4e85e1a0e841 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:35.641982 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e95e3-6d92-4980-8f26-4e85e1a0e841 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:35.646214 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e95e3-6d92-4980-8f26-4e85e1a0e841 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:35.704087 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1373/2750] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:07:35 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:07:35.709390 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d89935-7605-4c8a-945f-ca51c3ff6337 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:35.723159 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d89935-7605-4c8a-945f-ca51c3ff6337 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:35.868787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1267600-0c73-4202-b3c3-46e97b450098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:35.869779 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1267600-0c73-4202-b3c3-46e97b450098 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:35.870223 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1267600-0c73-4202-b3c3-46e97b450098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:35.871242 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1267600-0c73-4202-b3c3-46e97b450098 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:35.872129 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1267600-0c73-4202-b3c3-46e97b450098 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:35.879744 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1267600-0c73-4202-b3c3-46e97b450098 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:35.931751 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1378/2751] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:07:35 2022] POST /identity/v3/users => generated 287 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:07:35.936079 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e95e3-6d92-4980-8f26-4e85e1a0e841 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:35.952417 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e95e3-6d92-4980-8f26-4e85e1a0e841 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:35.967478 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1b3e7-f8f6-4922-bbdb-2798c0e39b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:35.968280 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1b3e7-f8f6-4922-bbdb-2798c0e39b34 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:35.968562 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1b3e7-f8f6-4922-bbdb-2798c0e39b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:35.968843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1b3e7-f8f6-4922-bbdb-2798c0e39b34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:35.969602 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab1b3e7-f8f6-4922-bbdb-2798c0e39b34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:35.973756 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab1b3e7-f8f6-4922-bbdb-2798c0e39b34 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:35.996737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1374/2752] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:07:36.000936 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1267600-0c73-4202-b3c3-46e97b450098 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:36.018201 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1267600-0c73-4202-b3c3-46e97b450098 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:36.032541 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177d311-330f-4c03-90a0-c03974bb0185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:36.033593 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177d311-330f-4c03-90a0-c03974bb0185 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:36.034066 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177d311-330f-4c03-90a0-c03974bb0185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:36.034541 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177d311-330f-4c03-90a0-c03974bb0185 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:36.035453 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f177d311-330f-4c03-90a0-c03974bb0185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:36.037337 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f177d311-330f-4c03-90a0-c03974bb0185 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:36.055144 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:36.070136 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1379/2753] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:07:36.081645 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1b3e7-f8f6-4922-bbdb-2798c0e39b34 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:36.101888 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1b3e7-f8f6-4922-bbdb-2798c0e39b34 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:36.119329 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68cd0e-30fb-4de9-844d-09e908e4983f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:36.120730 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68cd0e-30fb-4de9-844d-09e908e4983f admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:36.121004 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68cd0e-30fb-4de9-844d-09e908e4983f admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:36.121284 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68cd0e-30fb-4de9-844d-09e908e4983f admin admin] PATH_INFO: `/v3/projects/87cfccb52dd2434e956c45ea105e3d96/users/da906a9c8a4d403eb67d063e53830dae/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:36.153788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b68cd0e-30fb-4de9-844d-09e908e4983f admin admin] RBAC: Authorizing `identity:create_grant(project_id=87cfccb52dd2434e956c45ea105e3d96, user_id=da906a9c8a4d403eb67d063e53830dae, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:36.156541 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b68cd0e-30fb-4de9-844d-09e908e4983f admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:36.198410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1375/2754] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:07:36 2022] PUT /identity/v3/projects/87cfccb52dd2434e956c45ea105e3d96/users/da906a9c8a4d403eb67d063e53830dae/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:36.200705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e433f-107c-49ad-b863-a0b668caae62 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:36.202727 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e433f-107c-49ad-b863-a0b668caae62 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:36.203274 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e433f-107c-49ad-b863-a0b668caae62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:36.270709 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6d8e433f-107c-49ad-b863-a0b668caae62 None None] MFA Rules not processed for user `da906a9c8a4d403eb67d063e53830dae`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:36.272852 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6d8e433f-107c-49ad-b863-a0b668caae62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:36.452307 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1380/2755] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:36.458888 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68cd0e-30fb-4de9-844d-09e908e4983f admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:36.464813 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-7b68cd0e-30fb-4de9-844d-09e908e4983f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:36.610350 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68cd0e-30fb-4de9-844d-09e908e4983f admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:36.625373 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a6dbc-c6ff-48dc-ab50-6daa39a1b53a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:36.626434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a6dbc-c6ff-48dc-ab50-6daa39a1b53a service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:36.626933 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a6dbc-c6ff-48dc-ab50-6daa39a1b53a service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:36.627388 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a6dbc-c6ff-48dc-ab50-6daa39a1b53a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:36.642206 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a6dbc-c6ff-48dc-ab50-6daa39a1b53a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:36.644243 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a6dbc-c6ff-48dc-ab50-6daa39a1b53a service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:36.664921 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1376/2756] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:07:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:07:36.720185 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e433f-107c-49ad-b863-a0b668caae62 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:36.851430 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e433f-107c-49ad-b863-a0b668caae62 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:36.857978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5714a-520f-4597-a12a-cd866a598e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:36.858795 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5714a-520f-4597-a12a-cd866a598e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:36.859229 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5714a-520f-4597-a12a-cd866a598e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:36.859662 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5714a-520f-4597-a12a-cd866a598e50 admin admin] PATH_INFO: `/v3/users/da906a9c8a4d403eb67d063e53830dae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:36.861112 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5714a-520f-4597-a12a-cd866a598e50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da906a9c8a4d403eb67d063e53830dae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:36.863307 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5714a-520f-4597-a12a-cd866a598e50 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:36.980980 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-46f5714a-520f-4597-a12a-cd866a598e50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da906a9c8a4d403eb67d063e53830dae'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:36.995425 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-46f5714a-520f-4597-a12a-cd866a598e50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da906a9c8a4d403eb67d063e53830dae'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:36.997137 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-46f5714a-520f-4597-a12a-cd866a598e50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da906a9c8a4d403eb67d063e53830dae'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:37.016265 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-46f5714a-520f-4597-a12a-cd866a598e50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da906a9c8a4d403eb67d063e53830dae'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:37.017895 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-46f5714a-520f-4597-a12a-cd866a598e50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da906a9c8a4d403eb67d063e53830dae'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:37.082595 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1381/2757] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:07:36 2022] DELETE /identity/v3/users/da906a9c8a4d403eb67d063e53830dae => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:37.087409 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9910ee-e987-4712-be2d-742556f18beb None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:37.087737 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9910ee-e987-4712-be2d-742556f18beb None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:37.088036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9910ee-e987-4712-be2d-742556f18beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:37.128044 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-9a9910ee-e987-4712-be2d-742556f18beb None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:37.130375 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-9a9910ee-e987-4712-be2d-742556f18beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:37.303170 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1377/2758] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:37.319099 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5714a-520f-4597-a12a-cd866a598e50 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:37.320762 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-46f5714a-520f-4597-a12a-cd866a598e50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:37.348100 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:37.436668 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5714a-520f-4597-a12a-cd866a598e50 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:37.442240 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab038bf6-7f40-4276-9437-f3bce950b0fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:37.443221 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab038bf6-7f40-4276-9437-f3bce950b0fd service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:37.443685 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab038bf6-7f40-4276-9437-f3bce950b0fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:37.444104 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab038bf6-7f40-4276-9437-f3bce950b0fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:37.455468 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab038bf6-7f40-4276-9437-f3bce950b0fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:37.457712 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab038bf6-7f40-4276-9437-f3bce950b0fd service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:37.471592 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1382/2759] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:07:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:07:38.165661 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9910ee-e987-4712-be2d-742556f18beb None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:38.167738 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-9a9910ee-e987-4712-be2d-742556f18beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:38.191604 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9910ee-e987-4712-be2d-742556f18beb None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:38.197955 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89217258-d8bb-49f8-bd42-026a6f8e1b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:38.198964 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89217258-d8bb-49f8-bd42-026a6f8e1b60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:38.199838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89217258-d8bb-49f8-bd42-026a6f8e1b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:38.200305 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89217258-d8bb-49f8-bd42-026a6f8e1b60 admin admin] PATH_INFO: `/v3/projects/87cfccb52dd2434e956c45ea105e3d96` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:38.201761 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89217258-d8bb-49f8-bd42-026a6f8e1b60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87cfccb52dd2434e956c45ea105e3d96)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:38.204540 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89217258-d8bb-49f8-bd42-026a6f8e1b60 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:38.309195 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:38.324618 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-89217258-d8bb-49f8-bd42-026a6f8e1b60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87cfccb52dd2434e956c45ea105e3d96', 'request_id': 'req-89217258-d8bb-49f8-bd42-026a6f8e1b60'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:38.351125 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-89217258-d8bb-49f8-bd42-026a6f8e1b60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87cfccb52dd2434e956c45ea105e3d96', 'request_id': 'req-89217258-d8bb-49f8-bd42-026a6f8e1b60'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:38.377815 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-89217258-d8bb-49f8-bd42-026a6f8e1b60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87cfccb52dd2434e956c45ea105e3d96', 'request_id': 'req-89217258-d8bb-49f8-bd42-026a6f8e1b60'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:38.417522 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-89217258-d8bb-49f8-bd42-026a6f8e1b60 admin admin] The token cache is being invalidate because project 87cfccb52dd2434e956c45ea105e3d96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:07:38.417937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-89217258-d8bb-49f8-bd42-026a6f8e1b60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:38.418731 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-89217258-d8bb-49f8-bd42-026a6f8e1b60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:07:38.422790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1378/2760] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:07:38 2022] DELETE /identity/v3/projects/87cfccb52dd2434e956c45ea105e3d96 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:38.427924 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2317ae-b195-4bc9-9775-91561dc83d4c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:38.428218 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2317ae-b195-4bc9-9775-91561dc83d4c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:38.428661 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2317ae-b195-4bc9-9775-91561dc83d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:38.475508 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fe2317ae-b195-4bc9-9775-91561dc83d4c None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:38.479997 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fe2317ae-b195-4bc9-9775-91561dc83d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:38.649324 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1383/2761] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:38.651199 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3f1b1-2c49-45f2-a480-a421c0ca236b None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:38.651892 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3f1b1-2c49-45f2-a480-a421c0ca236b None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:38.652406 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3f1b1-2c49-45f2-a480-a421c0ca236b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:38.703291 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-7cf3f1b1-2c49-45f2-a480-a421c0ca236b None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:38.705922 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-7cf3f1b1-2c49-45f2-a480-a421c0ca236b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:38.719132 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1379/2762] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:38.723881 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2317ae-b195-4bc9-9775-91561dc83d4c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:38.739573 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2317ae-b195-4bc9-9775-91561dc83d4c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:38.748629 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646d026-51ff-4328-a980-c95817f515b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:38.750067 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646d026-51ff-4328-a980-c95817f515b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:38.750365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646d026-51ff-4328-a980-c95817f515b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:38.750656 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646d026-51ff-4328-a980-c95817f515b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:38.751394 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1646d026-51ff-4328-a980-c95817f515b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:07:38.759579 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1646d026-51ff-4328-a980-c95817f515b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:38.761380 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1646d026-51ff-4328-a980-c95817f515b3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:38.797068 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3f1b1-2c49-45f2-a480-a421c0ca236b None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:38.802159 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1384/2763] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:07:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:07:38.812933 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3f1b1-2c49-45f2-a480-a421c0ca236b None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:38.825514 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed586f89-2f9e-450b-97da-a98a4dae967e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:38.827666 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed586f89-2f9e-450b-97da-a98a4dae967e admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:38.827958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed586f89-2f9e-450b-97da-a98a4dae967e admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:38.828239 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed586f89-2f9e-450b-97da-a98a4dae967e admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:38.828937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed586f89-2f9e-450b-97da-a98a4dae967e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:38.832037 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed586f89-2f9e-450b-97da-a98a4dae967e admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:38.885944 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1380/2764] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:07:38 2022] POST /identity/v3/projects => generated 371 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:07:38.889869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646d026-51ff-4328-a980-c95817f515b3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:38.905627 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646d026-51ff-4328-a980-c95817f515b3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:39.032705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec619980-9de4-46f5-a465-5fbf6afd9a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:39.033283 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec619980-9de4-46f5-a465-5fbf6afd9a7b admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:39.033570 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec619980-9de4-46f5-a465-5fbf6afd9a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:39.033841 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec619980-9de4-46f5-a465-5fbf6afd9a7b admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:39.034696 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec619980-9de4-46f5-a465-5fbf6afd9a7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:39.037033 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec619980-9de4-46f5-a465-5fbf6afd9a7b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:39.078501 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed586f89-2f9e-450b-97da-a98a4dae967e admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:39.086713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1385/2765] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:07:38 2022] POST /identity/v3/users => generated 286 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:07:39.098471 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed586f89-2f9e-450b-97da-a98a4dae967e admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:39.105733 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aec361-5820-477a-8ddf-616a0d37eaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:39.107420 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aec361-5820-477a-8ddf-616a0d37eaef admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:39.107843 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aec361-5820-477a-8ddf-616a0d37eaef admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:39.108231 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aec361-5820-477a-8ddf-616a0d37eaef admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:39.109636 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aec361-5820-477a-8ddf-616a0d37eaef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:39.112299 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aec361-5820-477a-8ddf-616a0d37eaef admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:39.136899 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec619980-9de4-46f5-a465-5fbf6afd9a7b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:39.145960 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1381/2766] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:07:39.157570 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec619980-9de4-46f5-a465-5fbf6afd9a7b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:39.162635 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c1bfc-4eb0-4135-908f-a1e180f2469b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:39.163284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c1bfc-4eb0-4135-908f-a1e180f2469b admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:39.163589 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c1bfc-4eb0-4135-908f-a1e180f2469b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:39.163884 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c1bfc-4eb0-4135-908f-a1e180f2469b admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:39.164637 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c1bfc-4eb0-4135-908f-a1e180f2469b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:39.166259 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c1bfc-4eb0-4135-908f-a1e180f2469b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:39.183853 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1386/2767] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:07:39.186265 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aec361-5820-477a-8ddf-616a0d37eaef admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:39.203364 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aec361-5820-477a-8ddf-616a0d37eaef admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:39.208725 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9f78e-1be9-43f3-89f0-8a27c41d4128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:39.209505 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9f78e-1be9-43f3-89f0-8a27c41d4128 admin admin] REQUEST_METHOD: `PUT` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:39.209907 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9f78e-1be9-43f3-89f0-8a27c41d4128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:39.210201 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9f78e-1be9-43f3-89f0-8a27c41d4128 admin admin] PATH_INFO: `/v3/projects/666f12ad571f45c096947d0497291842/users/bca4c30c05b1498f861242024880f2e4/roles/7519356859fb40508c25bec8161207fd` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:39.228253 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9f78e-1be9-43f3-89f0-8a27c41d4128 admin admin] RBAC: Authorizing `identity:create_grant(project_id=666f12ad571f45c096947d0497291842, user_id=bca4c30c05b1498f861242024880f2e4, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:39.231007 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9f78e-1be9-43f3-89f0-8a27c41d4128 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:39.253445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1382/2768] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:07:39 2022] PUT /identity/v3/projects/666f12ad571f45c096947d0497291842/users/bca4c30c05b1498f861242024880f2e4/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:39.256542 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80818a98-02bb-4d61-b9c5-02881ff37715 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:39.256838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80818a98-02bb-4d61-b9c5-02881ff37715 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:39.257115 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80818a98-02bb-4d61-b9c5-02881ff37715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:39.284722 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-80818a98-02bb-4d61-b9c5-02881ff37715 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:39.285951 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-80818a98-02bb-4d61-b9c5-02881ff37715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:39.309571 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:39.385767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1387/2769] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:39.396527 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9f78e-1be9-43f3-89f0-8a27c41d4128 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:39.399218 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-80d9f78e-1be9-43f3-89f0-8a27c41d4128 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:39.483421 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9f78e-1be9-43f3-89f0-8a27c41d4128 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:39.487244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b455af-bcc0-45f2-9d1f-359ba9ab9ef2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:39.488165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b455af-bcc0-45f2-9d1f-359ba9ab9ef2 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:39.488523 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b455af-bcc0-45f2-9d1f-359ba9ab9ef2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:39.488807 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b455af-bcc0-45f2-9d1f-359ba9ab9ef2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:39.500986 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b455af-bcc0-45f2-9d1f-359ba9ab9ef2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:39.502459 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b455af-bcc0-45f2-9d1f-359ba9ab9ef2 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:39.515071 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1383/2770] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:07:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:07:44.639665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd011f-feeb-4fda-babc-b6896ca3c29e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:44.640161 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd011f-feeb-4fda-babc-b6896ca3c29e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:44.640263 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd011f-feeb-4fda-babc-b6896ca3c29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:44.679638 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-54fd011f-feeb-4fda-babc-b6896ca3c29e None None] MFA Rules not processed for user `bca4c30c05b1498f861242024880f2e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:44.681245 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-54fd011f-feeb-4fda-babc-b6896ca3c29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:44.812914 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1388/2771] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:44.817690 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b455af-bcc0-45f2-9d1f-359ba9ab9ef2 service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:44.835718 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b455af-bcc0-45f2-9d1f-359ba9ab9ef2 service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:44.844910 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501af84c-30c3-4d9d-83d0-d070600f67a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:44.845594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501af84c-30c3-4d9d-83d0-d070600f67a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:44.846032 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501af84c-30c3-4d9d-83d0-d070600f67a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:44.846312 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501af84c-30c3-4d9d-83d0-d070600f67a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:44.846982 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501af84c-30c3-4d9d-83d0-d070600f67a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:44.849158 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501af84c-30c3-4d9d-83d0-d070600f67a4 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:44.891517 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1384/2772] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:07:44 2022] POST /identity/v3/projects => generated 371 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:07:44.895786 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd011f-feeb-4fda-babc-b6896ca3c29e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:44.916469 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd011f-feeb-4fda-babc-b6896ca3c29e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:45.019078 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9164e15-dd14-4fbe-9ac0-bd846b0408a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:45.019778 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9164e15-dd14-4fbe-9ac0-bd846b0408a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:45.020082 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9164e15-dd14-4fbe-9ac0-bd846b0408a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:45.020347 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9164e15-dd14-4fbe-9ac0-bd846b0408a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:45.021027 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9164e15-dd14-4fbe-9ac0-bd846b0408a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:45.023008 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9164e15-dd14-4fbe-9ac0-bd846b0408a5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:45.058396 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1389/2773] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:07:44 2022] POST /identity/v3/users => generated 292 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:07:45.060795 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501af84c-30c3-4d9d-83d0-d070600f67a4 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:45.073194 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501af84c-30c3-4d9d-83d0-d070600f67a4 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:45.082234 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba86fc2-34f0-435e-9fa5-b32e381ea0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:45.083140 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba86fc2-34f0-435e-9fa5-b32e381ea0b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:45.083608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba86fc2-34f0-435e-9fa5-b32e381ea0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:45.084182 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba86fc2-34f0-435e-9fa5-b32e381ea0b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:45.085202 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba86fc2-34f0-435e-9fa5-b32e381ea0b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:45.087830 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba86fc2-34f0-435e-9fa5-b32e381ea0b4 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:45.107697 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1385/2774] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:07:45.108724 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9164e15-dd14-4fbe-9ac0-bd846b0408a5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:45.120541 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9164e15-dd14-4fbe-9ac0-bd846b0408a5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:45.130264 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c4578-2024-47e5-a9a0-ecf73a2e45e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:45.131386 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c4578-2024-47e5-a9a0-ecf73a2e45e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:45.131920 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c4578-2024-47e5-a9a0-ecf73a2e45e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:45.132426 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c4578-2024-47e5-a9a0-ecf73a2e45e6 admin admin] PATH_INFO: `/v3/projects/a6e3bf6c6e9641e58d936dd8e15550ff/users/189407887dd343ecac794fcbf7cbf187/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:45.154318 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c4578-2024-47e5-a9a0-ecf73a2e45e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6e3bf6c6e9641e58d936dd8e15550ff, user_id=189407887dd343ecac794fcbf7cbf187, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:45.157365 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c4578-2024-47e5-a9a0-ecf73a2e45e6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:45.179879 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1390/2775] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:07:45 2022] PUT /identity/v3/projects/a6e3bf6c6e9641e58d936dd8e15550ff/users/189407887dd343ecac794fcbf7cbf187/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:45.185172 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba86fc2-34f0-435e-9fa5-b32e381ea0b4 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:45.199105 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba86fc2-34f0-435e-9fa5-b32e381ea0b4 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:45.252363 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:45.284603 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32c13a1-52e1-45a5-8a60-93948ceba895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:45.285401 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c13a1-52e1-45a5-8a60-93948ceba895 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:45.286038 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c13a1-52e1-45a5-8a60-93948ceba895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:45.286377 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c13a1-52e1-45a5-8a60-93948ceba895 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:45.287150 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32c13a1-52e1-45a5-8a60-93948ceba895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:45.288874 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32c13a1-52e1-45a5-8a60-93948ceba895 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:45.300567 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1386/2776] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:07:45.304814 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c4578-2024-47e5-a9a0-ecf73a2e45e6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:45.317824 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c4578-2024-47e5-a9a0-ecf73a2e45e6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:45.322821 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc03ddd-d1dd-466d-ac8a-d450a5d63d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:45.323685 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc03ddd-d1dd-466d-ac8a-d450a5d63d9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:45.324211 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc03ddd-d1dd-466d-ac8a-d450a5d63d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:45.324599 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc03ddd-d1dd-466d-ac8a-d450a5d63d9c admin admin] PATH_INFO: `/v3/projects/a6e3bf6c6e9641e58d936dd8e15550ff/users/189407887dd343ecac794fcbf7cbf187/roles/72bf8bd23d2c4b76ad32b7b35d6db412` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:45.326703 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc03ddd-d1dd-466d-ac8a-d450a5d63d9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6e3bf6c6e9641e58d936dd8e15550ff, user_id=189407887dd343ecac794fcbf7cbf187, role_id=72bf8bd23d2c4b76ad32b7b35d6db412)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:45.329482 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc03ddd-d1dd-466d-ac8a-d450a5d63d9c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:45.339374 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:45.347237 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1391/2777] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:07:45 2022] PUT /identity/v3/projects/a6e3bf6c6e9641e58d936dd8e15550ff/users/189407887dd343ecac794fcbf7cbf187/roles/72bf8bd23d2c4b76ad32b7b35d6db412 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:07:47.918518 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad8455-d5b6-4d2b-b99d-ecf6d26a87f4 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:47.919039 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad8455-d5b6-4d2b-b99d-ecf6d26a87f4 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:47.919039 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad8455-d5b6-4d2b-b99d-ecf6d26a87f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:47.939184 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-85ad8455-d5b6-4d2b-b99d-ecf6d26a87f4 None None] MFA Rules not processed for user `c1e8fc3655db4d428bde11a8fc70c0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:47.940903 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-85ad8455-d5b6-4d2b-b99d-ecf6d26a87f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:47.998539 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1387/2778] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:50.718284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2e673-1029-4f57-874b-6035efa935a6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:50.720349 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2e673-1029-4f57-874b-6035efa935a6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:50.720816 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2e673-1029-4f57-874b-6035efa935a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:50.757851 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-13c2e673-1029-4f57-874b-6035efa935a6 None None] MFA Rules not processed for user `3a8f3eb811ae40ad9cb64261f95a992e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:50.760268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-13c2e673-1029-4f57-874b-6035efa935a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:50.920748 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1392/2779] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jul 26 05:07:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:50.936342 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad8455-d5b6-4d2b-b99d-ecf6d26a87f4 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:50.942730 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-85ad8455-d5b6-4d2b-b99d-ecf6d26a87f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:51.093617 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01cf5d6-6d36-48a0-b6e4-f232c1bc9ceb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:51.094097 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01cf5d6-6d36-48a0-b6e4-f232c1bc9ceb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:51.094444 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01cf5d6-6d36-48a0-b6e4-f232c1bc9ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:51.124317 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad8455-d5b6-4d2b-b99d-ecf6d26a87f4 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:51.137214 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a01cf5d6-6d36-48a0-b6e4-f232c1bc9ceb None None] MFA Rules not processed for user `189407887dd343ecac794fcbf7cbf187`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:51.138608 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a01cf5d6-6d36-48a0-b6e4-f232c1bc9ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:51.148980 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c83c8-28cf-4d9d-8b5e-38f7021a7271 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5218319aaa5424882714bffcbeee730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:51.150858 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c83c8-28cf-4d9d-8b5e-38f7021a7271 service swift] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:51.151143 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c83c8-28cf-4d9d-8b5e-38f7021a7271 service swift] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:51.151350 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c83c8-28cf-4d9d-8b5e-38f7021a7271 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:51.177187 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c83c8-28cf-4d9d-8b5e-38f7021a7271 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:51.178968 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c83c8-28cf-4d9d-8b5e-38f7021a7271 service swift] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:51.192050 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1388/2780] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jul 26 05:07:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 05:07:51.275321 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1393/2781] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:51 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:07:51.283420 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c83c8-28cf-4d9d-8b5e-38f7021a7271 service swift] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:07:51.284928 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-b49c83c8-28cf-4d9d-8b5e-38f7021a7271 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:51.447758 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c83c8-28cf-4d9d-8b5e-38f7021a7271 service swift] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:07:51.466379 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c314eb-828e-41cb-afa4-750e7df99cd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:07:51.467084 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c314eb-828e-41cb-afa4-750e7df99cd4 service nova] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:51.467381 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c314eb-828e-41cb-afa4-750e7df99cd4 service nova] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:51.467683 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c314eb-828e-41cb-afa4-750e7df99cd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:51.477442 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:07:51.665036 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c314eb-828e-41cb-afa4-750e7df99cd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:07:51.667134 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c314eb-828e-41cb-afa4-750e7df99cd4 service nova] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:07:51.697617 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1389/2782] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:07:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:07:54.441394 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad8805-a32a-4f83-a672-130f4f553178 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:07:54.442006 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad8805-a32a-4f83-a672-130f4f553178 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:07:54.442006 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad8805-a32a-4f83-a672-130f4f553178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:07:54.500220 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-27ad8805-a32a-4f83-a672-130f4f553178 None None] MFA Rules not processed for user `bca4c30c05b1498f861242024880f2e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:07:54.501595 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-27ad8805-a32a-4f83-a672-130f4f553178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:07:54.529241 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1394/2783] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:07:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:08:01.683164 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=25 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:08:04.527169 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=74 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:08:13.765431 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c314eb-828e-41cb-afa4-750e7df99cd4 service nova] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:08:13.767119 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-33c314eb-828e-41cb-afa4-750e7df99cd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:08:13.869973 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c314eb-828e-41cb-afa4-750e7df99cd4 service nova] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:08:13.873741 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522f82f-982a-42e8-84a7-64c6722d3ccc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:08:13.874370 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522f82f-982a-42e8-84a7-64c6722d3ccc service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:08:13.874649 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522f82f-982a-42e8-84a7-64c6722d3ccc service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:08:13.874920 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522f82f-982a-42e8-84a7-64c6722d3ccc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:08:13.879360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-0522f82f-982a-42e8-84a7-64c6722d3ccc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:08:13.891434 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0522f82f-982a-42e8-84a7-64c6722d3ccc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:08:13.892968 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0522f82f-982a-42e8-84a7-64c6722d3ccc service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:08:13.904769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1390/2784] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:08:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:08:14.923379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad8805-a32a-4f83-a672-130f4f553178 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:08:14.925157 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-27ad8805-a32a-4f83-a672-130f4f553178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:08:15.030105 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad8805-a32a-4f83-a672-130f4f553178 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:08:15.035195 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2719c3f-865e-47cd-a4f3-41fd4f1e127b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5049dba7ffde47b8af3e6fd55b5703e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:08:15.035849 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2719c3f-865e-47cd-a4f3-41fd4f1e127b service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:08:15.036133 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2719c3f-865e-47cd-a4f3-41fd4f1e127b service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:08:15.036409 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2719c3f-865e-47cd-a4f3-41fd4f1e127b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:08:15.038211 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a2719c3f-865e-47cd-a4f3-41fd4f1e127b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:08:15.050583 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2719c3f-865e-47cd-a4f3-41fd4f1e127b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:08:15.052201 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2719c3f-865e-47cd-a4f3-41fd4f1e127b service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:08:15.063787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1395/2785] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:08:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:08:15.809634 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522f82f-982a-42e8-84a7-64c6722d3ccc service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:08:15.811356 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-0522f82f-982a-42e8-84a7-64c6722d3ccc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:08:15.879918 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522f82f-982a-42e8-84a7-64c6722d3ccc service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:08:15.884205 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5acdfa-f245-46e1-97fd-eeece1034b88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:08:15.884839 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5acdfa-f245-46e1-97fd-eeece1034b88 service cinder] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:08:15.885124 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5acdfa-f245-46e1-97fd-eeece1034b88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:08:15.885403 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5acdfa-f245-46e1-97fd-eeece1034b88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:08:15.887278 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-5f5acdfa-f245-46e1-97fd-eeece1034b88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:08:15.959526 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5acdfa-f245-46e1-97fd-eeece1034b88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:08:15.961045 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5acdfa-f245-46e1-97fd-eeece1034b88 service cinder] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:08:15.971150 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1391/2786] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:08:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:08:16.106471 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021bd71-51f6-4e6a-adac-55f26c8362fe None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:08:16.106974 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021bd71-51f6-4e6a-adac-55f26c8362fe None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:08:16.107076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021bd71-51f6-4e6a-adac-55f26c8362fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:08:16.128371 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2021bd71-51f6-4e6a-adac-55f26c8362fe None None] MFA Rules not processed for user `148ea5a9b7a74172b290696be99aa5c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:08:16.130369 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2021bd71-51f6-4e6a-adac-55f26c8362fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:08:16.137763 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1396/2787] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:08:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:08:25.694168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756d9e6-9e6f-431d-80f7-0e7fd535ff37 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:08:25.694706 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756d9e6-9e6f-431d-80f7-0e7fd535ff37 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:08:25.694706 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756d9e6-9e6f-431d-80f7-0e7fd535ff37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:08:25.715110 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-4756d9e6-9e6f-431d-80f7-0e7fd535ff37 None None] MFA Rules not processed for user `92b9d212b86348c6b51a6b02818a0e80`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:08:25.716372 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-4756d9e6-9e6f-431d-80f7-0e7fd535ff37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:08:25.729395 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:08:25.777638 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1392/2788] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:08:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:08:26.122330 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=47 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:08:31.631870 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021bd71-51f6-4e6a-adac-55f26c8362fe None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:08:31.641648 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021bd71-51f6-4e6a-adac-55f26c8362fe None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:08:31.645410 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314da07-2a71-4fe2-9eb4-880a24919b5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:08:31.646117 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314da07-2a71-4fe2-9eb4-880a24919b5d service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:08:31.646451 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314da07-2a71-4fe2-9eb4-880a24919b5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:08:31.646789 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314da07-2a71-4fe2-9eb4-880a24919b5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:08:31.648622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2314da07-2a71-4fe2-9eb4-880a24919b5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:08:31.716694 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2314da07-2a71-4fe2-9eb4-880a24919b5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:08:31.718728 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2314da07-2a71-4fe2-9eb4-880a24919b5d service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:08:31.730603 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1397/2789] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:08:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:08:33.574532 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756d9e6-9e6f-431d-80f7-0e7fd535ff37 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:08:33.585320 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756d9e6-9e6f-431d-80f7-0e7fd535ff37 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:08:33.589691 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a980c-96a7-4cf7-af34-8a44a3fb1d05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:08:33.590275 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a980c-96a7-4cf7-af34-8a44a3fb1d05 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:08:33.590536 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a980c-96a7-4cf7-af34-8a44a3fb1d05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:08:33.590816 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a980c-96a7-4cf7-af34-8a44a3fb1d05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:08:33.603541 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a980c-96a7-4cf7-af34-8a44a3fb1d05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:08:33.605289 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a980c-96a7-4cf7-af34-8a44a3fb1d05 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:08:33.617378 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1393/2790] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:08:33 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:08:41.725056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=30 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:08:43.611394 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=23 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:09:39.137231 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fe7f6-c4ba-43e5-a26d-8e8a22e0c472 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:09:39.138013 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fe7f6-c4ba-43e5-a26d-8e8a22e0c472 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:09:39.138013 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fe7f6-c4ba-43e5-a26d-8e8a22e0c472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:09:39.157992 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-911fe7f6-c4ba-43e5-a26d-8e8a22e0c472 None None] MFA Rules not processed for user `92b9d212b86348c6b51a6b02818a0e80`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:09:39.159236 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-911fe7f6-c4ba-43e5-a26d-8e8a22e0c472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:09:39.166776 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1398/2791] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:09:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:09:49.152211 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:09:58.894240 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a980c-96a7-4cf7-af34-8a44a3fb1d05 service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:09:58.904154 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a980c-96a7-4cf7-af34-8a44a3fb1d05 service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:09:58.908633 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f6d24e-bfe5-4b9d-b413-f33db93f5664 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:09:58.909251 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6d24e-bfe5-4b9d-b413-f33db93f5664 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:09:58.909525 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6d24e-bfe5-4b9d-b413-f33db93f5664 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:09:58.909793 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6d24e-bfe5-4b9d-b413-f33db93f5664 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:09:58.911857 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-81f6d24e-bfe5-4b9d-b413-f33db93f5664 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:09:58.947591 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f6d24e-bfe5-4b9d-b413-f33db93f5664 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:09:58.949113 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f6d24e-bfe5-4b9d-b413-f33db93f5664 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:09:58.970064 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1394/2792] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:09:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:10:04.680181 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fe7f6-c4ba-43e5-a26d-8e8a22e0c472 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:10:04.692335 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fe7f6-c4ba-43e5-a26d-8e8a22e0c472 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:10:04.697456 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928cd49-f992-4fa8-a2f1-6ec69da7ac01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:10:04.698085 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928cd49-f992-4fa8-a2f1-6ec69da7ac01 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:10:04.698370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928cd49-f992-4fa8-a2f1-6ec69da7ac01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:10:04.698647 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928cd49-f992-4fa8-a2f1-6ec69da7ac01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:10:04.700996 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9928cd49-f992-4fa8-a2f1-6ec69da7ac01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:10:04.715096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9928cd49-f992-4fa8-a2f1-6ec69da7ac01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:10:04.716566 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9928cd49-f992-4fa8-a2f1-6ec69da7ac01 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:10:04.727670 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1399/2793] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:10:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:10:07.288304 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c61a9f-2503-4639-868f-d89422bd71c2 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:10:07.288767 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c61a9f-2503-4639-868f-d89422bd71c2 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:10:07.288872 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c61a9f-2503-4639-868f-d89422bd71c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:10:07.318533 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-95c61a9f-2503-4639-868f-d89422bd71c2 None None] MFA Rules not processed for user `3a8f3eb811ae40ad9cb64261f95a992e`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:10:07.319781 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-95c61a9f-2503-4639-868f-d89422bd71c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:10:07.327579 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1395/2794] 10.222.0.17 () {58 vars in 967 bytes} [Tue Jul 26 05:10:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:10:07.368301 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928cd49-f992-4fa8-a2f1-6ec69da7ac01 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:10:07.380188 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928cd49-f992-4fa8-a2f1-6ec69da7ac01 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:10:07.385939 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8bc54-4571-456e-8b61-9a3a51b76292 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5218319aaa5424882714bffcbeee730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:10:07.386561 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8bc54-4571-456e-8b61-9a3a51b76292 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:10:07.386847 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8bc54-4571-456e-8b61-9a3a51b76292 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:10:07.387127 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8bc54-4571-456e-8b61-9a3a51b76292 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:10:07.396119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8bc54-4571-456e-8b61-9a3a51b76292 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:10:07.397735 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8bc54-4571-456e-8b61-9a3a51b76292 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:10:07.406192 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1400/2795] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jul 26 05:10:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 05:10:14.157565 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c61a9f-2503-4639-868f-d89422bd71c2 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:10:14.160982 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-95c61a9f-2503-4639-868f-d89422bd71c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:10:14.258600 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c61a9f-2503-4639-868f-d89422bd71c2 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:10:14.262951 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d52f58-c524-4937-9ea7-048604fda8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:10:14.263621 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d52f58-c524-4937-9ea7-048604fda8db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:10:14.263937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d52f58-c524-4937-9ea7-048604fda8db admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:10:14.264205 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d52f58-c524-4937-9ea7-048604fda8db admin admin] PATH_INFO: `/v3/users/c1e8fc3655db4d428bde11a8fc70c0dc` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:10:14.265372 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d52f58-c524-4937-9ea7-048604fda8db admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1e8fc3655db4d428bde11a8fc70c0dc)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:10:14.267292 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d52f58-c524-4937-9ea7-048604fda8db admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:10:14.318472 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-75d52f58-c524-4937-9ea7-048604fda8db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1e8fc3655db4d428bde11a8fc70c0dc'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:14.325389 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-75d52f58-c524-4937-9ea7-048604fda8db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1e8fc3655db4d428bde11a8fc70c0dc'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:14.334239 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-75d52f58-c524-4937-9ea7-048604fda8db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1e8fc3655db4d428bde11a8fc70c0dc'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:14.334887 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-75d52f58-c524-4937-9ea7-048604fda8db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1e8fc3655db4d428bde11a8fc70c0dc'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:14.335536 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-75d52f58-c524-4937-9ea7-048604fda8db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1e8fc3655db4d428bde11a8fc70c0dc'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:14.347423 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:10:14.355767 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1396/2796] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:10:14 2022] DELETE /identity/v3/users/c1e8fc3655db4d428bde11a8fc70c0dc => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:10:14.541394 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8bc54-4571-456e-8b61-9a3a51b76292 service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:10:14.552459 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8bc54-4571-456e-8b61-9a3a51b76292 service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:10:14.612573 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b0420-b75b-4cc9-8d0e-b1bb3d110a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:10:14.613473 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b0420-b75b-4cc9-8d0e-b1bb3d110a95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:10:14.613783 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b0420-b75b-4cc9-8d0e-b1bb3d110a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:10:14.614044 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b0420-b75b-4cc9-8d0e-b1bb3d110a95 admin admin] PATH_INFO: `/v3/projects/f1b92b49730c4ce7829e7f14b6870b43` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:10:14.615226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262b0420-b75b-4cc9-8d0e-b1bb3d110a95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1b92b49730c4ce7829e7f14b6870b43)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:10:14.617192 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262b0420-b75b-4cc9-8d0e-b1bb3d110a95 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:10:14.669056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-262b0420-b75b-4cc9-8d0e-b1bb3d110a95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1b92b49730c4ce7829e7f14b6870b43', 'request_id': 'req-262b0420-b75b-4cc9-8d0e-b1bb3d110a95'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:14.675681 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-262b0420-b75b-4cc9-8d0e-b1bb3d110a95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1b92b49730c4ce7829e7f14b6870b43', 'request_id': 'req-262b0420-b75b-4cc9-8d0e-b1bb3d110a95'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:14.686247 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-262b0420-b75b-4cc9-8d0e-b1bb3d110a95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1b92b49730c4ce7829e7f14b6870b43', 'request_id': 'req-262b0420-b75b-4cc9-8d0e-b1bb3d110a95'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:14.698058 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-262b0420-b75b-4cc9-8d0e-b1bb3d110a95 admin admin] The token cache is being invalidate because project f1b92b49730c4ce7829e7f14b6870b43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:10:14.698347 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-262b0420-b75b-4cc9-8d0e-b1bb3d110a95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:14.699038 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-262b0420-b75b-4cc9-8d0e-b1bb3d110a95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:14.701001 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1401/2797] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:10:14 2022] DELETE /identity/v3/projects/f1b92b49730c4ce7829e7f14b6870b43 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:10:14.704945 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d52f58-c524-4937-9ea7-048604fda8db admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:10:14.706440 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-75d52f58-c524-4937-9ea7-048604fda8db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:10:14.724075 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d52f58-c524-4937-9ea7-048604fda8db admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:10:14.782194 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f82b56-5ab3-46d4-8615-8c9e71e8a433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:10:14.782828 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f82b56-5ab3-46d4-8615-8c9e71e8a433 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:10:14.783104 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f82b56-5ab3-46d4-8615-8c9e71e8a433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:10:14.783371 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f82b56-5ab3-46d4-8615-8c9e71e8a433 admin admin] PATH_INFO: `/v3/users/434c5e94263e4f409ca4afdf093d3dde` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:10:14.784725 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f82b56-5ab3-46d4-8615-8c9e71e8a433 admin admin] RBAC: Authorizing `identity:delete_user(user_id=434c5e94263e4f409ca4afdf093d3dde)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:10:14.786639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f82b56-5ab3-46d4-8615-8c9e71e8a433 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:10:14.835990 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c4f82b56-5ab3-46d4-8615-8c9e71e8a433 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '434c5e94263e4f409ca4afdf093d3dde'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:14.843398 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c4f82b56-5ab3-46d4-8615-8c9e71e8a433 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '434c5e94263e4f409ca4afdf093d3dde'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:14.852917 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c4f82b56-5ab3-46d4-8615-8c9e71e8a433 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '434c5e94263e4f409ca4afdf093d3dde'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:14.853706 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c4f82b56-5ab3-46d4-8615-8c9e71e8a433 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '434c5e94263e4f409ca4afdf093d3dde'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:14.854446 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c4f82b56-5ab3-46d4-8615-8c9e71e8a433 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '434c5e94263e4f409ca4afdf093d3dde'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:14.871828 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1397/2798] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:10:14 2022] DELETE /identity/v3/users/434c5e94263e4f409ca4afdf093d3dde => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:10:15.052176 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b0420-b75b-4cc9-8d0e-b1bb3d110a95 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:10:15.053736 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-262b0420-b75b-4cc9-8d0e-b1bb3d110a95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:10:15.067255 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b0420-b75b-4cc9-8d0e-b1bb3d110a95 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:10:15.124967 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb84b81-f4d4-4d8b-825e-f127645549c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:10:15.125626 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb84b81-f4d4-4d8b-825e-f127645549c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:10:15.125911 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb84b81-f4d4-4d8b-825e-f127645549c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:10:15.126189 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb84b81-f4d4-4d8b-825e-f127645549c1 admin admin] PATH_INFO: `/v3/projects/838457cc16d44a1f8a2b951f2d5e402a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:10:15.127387 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb84b81-f4d4-4d8b-825e-f127645549c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=838457cc16d44a1f8a2b951f2d5e402a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:10:15.129420 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb84b81-f4d4-4d8b-825e-f127645549c1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:10:15.135569 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:10:15.178945 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3bb84b81-f4d4-4d8b-825e-f127645549c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '838457cc16d44a1f8a2b951f2d5e402a', 'request_id': 'req-3bb84b81-f4d4-4d8b-825e-f127645549c1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:15.185697 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3bb84b81-f4d4-4d8b-825e-f127645549c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '838457cc16d44a1f8a2b951f2d5e402a', 'request_id': 'req-3bb84b81-f4d4-4d8b-825e-f127645549c1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:15.194444 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3bb84b81-f4d4-4d8b-825e-f127645549c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '838457cc16d44a1f8a2b951f2d5e402a', 'request_id': 'req-3bb84b81-f4d4-4d8b-825e-f127645549c1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:15.204946 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3bb84b81-f4d4-4d8b-825e-f127645549c1 admin admin] The token cache is being invalidate because project 838457cc16d44a1f8a2b951f2d5e402a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:10:15.205269 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3bb84b81-f4d4-4d8b-825e-f127645549c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:15.205942 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3bb84b81-f4d4-4d8b-825e-f127645549c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:15.207817 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1402/2799] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:10:15 2022] DELETE /identity/v3/projects/838457cc16d44a1f8a2b951f2d5e402a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:10:24.868268 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=55,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:10:25.202143 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:10:42.151008 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f82b56-5ab3-46d4-8615-8c9e71e8a433 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:10:42.152565 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c4f82b56-5ab3-46d4-8615-8c9e71e8a433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:10:42.216101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f82b56-5ab3-46d4-8615-8c9e71e8a433 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:10:42.274031 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6145aa9b-58ed-4257-8a13-39911519e28f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:10:42.274677 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145aa9b-58ed-4257-8a13-39911519e28f service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:10:42.274964 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145aa9b-58ed-4257-8a13-39911519e28f service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:10:42.275238 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145aa9b-58ed-4257-8a13-39911519e28f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:10:42.277120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-6145aa9b-58ed-4257-8a13-39911519e28f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:10:42.359631 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6145aa9b-58ed-4257-8a13-39911519e28f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:10:42.361160 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6145aa9b-58ed-4257-8a13-39911519e28f service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:10:42.371654 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1398/2800] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:10:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 05:10:45.825539 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14424d-3f12-4243-b25b-e36c11dbc2bf None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:10:45.825993 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14424d-3f12-4243-b25b-e36c11dbc2bf None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:10:45.826285 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14424d-3f12-4243-b25b-e36c11dbc2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:10:45.851710 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6d14424d-3f12-4243-b25b-e36c11dbc2bf None None] MFA Rules not processed for user `92b9d212b86348c6b51a6b02818a0e80`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:10:45.853150 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6d14424d-3f12-4243-b25b-e36c11dbc2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:10:45.947508 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1403/2801] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:10:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:10:45.994349 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6145aa9b-58ed-4257-8a13-39911519e28f service neutron] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:10:45.996692 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-6145aa9b-58ed-4257-8a13-39911519e28f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:10:46.123318 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6145aa9b-58ed-4257-8a13-39911519e28f service neutron] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:10:46.128768 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f60a0-4415-4bda-8d79-db725e0fc3be service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5049dba7ffde47b8af3e6fd55b5703e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:10:46.129401 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f60a0-4415-4bda-8d79-db725e0fc3be service placement] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:10:46.129904 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f60a0-4415-4bda-8d79-db725e0fc3be service placement] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:10:46.130185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f60a0-4415-4bda-8d79-db725e0fc3be service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:10:46.132086 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-950f60a0-4415-4bda-8d79-db725e0fc3be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:10:46.142205 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:10:46.144603 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950f60a0-4415-4bda-8d79-db725e0fc3be service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:10:46.146828 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950f60a0-4415-4bda-8d79-db725e0fc3be service placement] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:10:46.157576 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1399/2802] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:10:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:10:51.143931 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14424d-3f12-4243-b25b-e36c11dbc2bf None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:10:51.146111 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6d14424d-3f12-4243-b25b-e36c11dbc2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:10:51.161393 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14424d-3f12-4243-b25b-e36c11dbc2bf None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:10:51.165888 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d7544-2d75-4024-948d-4dd318eb539f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:10:51.166541 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d7544-2d75-4024-948d-4dd318eb539f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:10:51.166998 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d7544-2d75-4024-948d-4dd318eb539f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:10:51.167315 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d7544-2d75-4024-948d-4dd318eb539f admin admin] PATH_INFO: `/v3/users/148ea5a9b7a74172b290696be99aa5c1` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:10:51.168588 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d7544-2d75-4024-948d-4dd318eb539f admin admin] RBAC: Authorizing `identity:delete_user(user_id=148ea5a9b7a74172b290696be99aa5c1)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:10:51.170796 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d7544-2d75-4024-948d-4dd318eb539f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:10:51.222163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a00d7544-2d75-4024-948d-4dd318eb539f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '148ea5a9b7a74172b290696be99aa5c1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:51.230599 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a00d7544-2d75-4024-948d-4dd318eb539f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '148ea5a9b7a74172b290696be99aa5c1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:51.231300 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a00d7544-2d75-4024-948d-4dd318eb539f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '148ea5a9b7a74172b290696be99aa5c1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:51.243126 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a00d7544-2d75-4024-948d-4dd318eb539f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '148ea5a9b7a74172b290696be99aa5c1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:51.243780 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a00d7544-2d75-4024-948d-4dd318eb539f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '148ea5a9b7a74172b290696be99aa5c1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:51.261580 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1404/2803] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:10:51 2022] DELETE /identity/v3/users/148ea5a9b7a74172b290696be99aa5c1 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:10:51.463284 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f60a0-4415-4bda-8d79-db725e0fc3be service placement] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:10:51.479428 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f60a0-4415-4bda-8d79-db725e0fc3be service placement] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:10:51.545453 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a75deb-7dc1-43ef-8645-081fde90d7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:10:51.547027 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a75deb-7dc1-43ef-8645-081fde90d7ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:10:51.547494 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a75deb-7dc1-43ef-8645-081fde90d7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:10:51.547831 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a75deb-7dc1-43ef-8645-081fde90d7ba admin admin] PATH_INFO: `/v3/projects/987757bb89e64419822e021b15a65c31` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:10:51.549694 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a75deb-7dc1-43ef-8645-081fde90d7ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=987757bb89e64419822e021b15a65c31)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:10:51.551542 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a75deb-7dc1-43ef-8645-081fde90d7ba admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:10:51.601884 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c0a75deb-7dc1-43ef-8645-081fde90d7ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '987757bb89e64419822e021b15a65c31', 'request_id': 'req-c0a75deb-7dc1-43ef-8645-081fde90d7ba'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:51.610290 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c0a75deb-7dc1-43ef-8645-081fde90d7ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '987757bb89e64419822e021b15a65c31', 'request_id': 'req-c0a75deb-7dc1-43ef-8645-081fde90d7ba'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:51.616810 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c0a75deb-7dc1-43ef-8645-081fde90d7ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '987757bb89e64419822e021b15a65c31', 'request_id': 'req-c0a75deb-7dc1-43ef-8645-081fde90d7ba'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:51.627191 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c0a75deb-7dc1-43ef-8645-081fde90d7ba admin admin] The token cache is being invalidate because project 987757bb89e64419822e021b15a65c31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:10:51.627462 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c0a75deb-7dc1-43ef-8645-081fde90d7ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:51.628138 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-c0a75deb-7dc1-43ef-8645-081fde90d7ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:51.629672 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1400/2804] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:10:51 2022] DELETE /identity/v3/projects/987757bb89e64419822e021b15a65c31 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:10:51.634614 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d7544-2d75-4024-948d-4dd318eb539f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:10:51.636174 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a00d7544-2d75-4024-948d-4dd318eb539f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:10:51.650040 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d7544-2d75-4024-948d-4dd318eb539f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:10:51.723346 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a21fd-e90e-432f-b28c-3ffe2ba6dc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:10:51.724487 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a21fd-e90e-432f-b28c-3ffe2ba6dc09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:10:51.724777 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a21fd-e90e-432f-b28c-3ffe2ba6dc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:10:51.725055 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a21fd-e90e-432f-b28c-3ffe2ba6dc09 admin admin] PATH_INFO: `/v3/users/c9817b6598dc480898e0b9f5c9ee5e5d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:10:51.726287 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a21fd-e90e-432f-b28c-3ffe2ba6dc09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9817b6598dc480898e0b9f5c9ee5e5d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:10:51.728361 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a21fd-e90e-432f-b28c-3ffe2ba6dc09 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:10:51.750402 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:10:51.784439 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-498a21fd-e90e-432f-b28c-3ffe2ba6dc09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9817b6598dc480898e0b9f5c9ee5e5d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:51.791089 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-498a21fd-e90e-432f-b28c-3ffe2ba6dc09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9817b6598dc480898e0b9f5c9ee5e5d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:51.791797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-498a21fd-e90e-432f-b28c-3ffe2ba6dc09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9817b6598dc480898e0b9f5c9ee5e5d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:51.800939 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-498a21fd-e90e-432f-b28c-3ffe2ba6dc09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9817b6598dc480898e0b9f5c9ee5e5d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:51.801625 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-498a21fd-e90e-432f-b28c-3ffe2ba6dc09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9817b6598dc480898e0b9f5c9ee5e5d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:51.820356 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1405/2805] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:10:51 2022] DELETE /identity/v3/users/c9817b6598dc480898e0b9f5c9ee5e5d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:10:51.998449 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a75deb-7dc1-43ef-8645-081fde90d7ba admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:10:52.000263 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c0a75deb-7dc1-43ef-8645-081fde90d7ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:10:52.020946 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a75deb-7dc1-43ef-8645-081fde90d7ba admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:10:52.089233 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b38b7d-5c23-463b-82e4-ffef27d6864f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:10:52.089901 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b38b7d-5c23-463b-82e4-ffef27d6864f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:10:52.090181 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b38b7d-5c23-463b-82e4-ffef27d6864f admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:10:52.090451 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b38b7d-5c23-463b-82e4-ffef27d6864f admin admin] PATH_INFO: `/v3/projects/6874e6654d734910b67db92475ece16e` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:10:52.091769 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b38b7d-5c23-463b-82e4-ffef27d6864f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6874e6654d734910b67db92475ece16e)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:10:52.093722 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b38b7d-5c23-463b-82e4-ffef27d6864f admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:10:52.126838 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:10:52.143093 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-e0b38b7d-5c23-463b-82e4-ffef27d6864f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6874e6654d734910b67db92475ece16e', 'request_id': 'req-e0b38b7d-5c23-463b-82e4-ffef27d6864f'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:52.154014 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-e0b38b7d-5c23-463b-82e4-ffef27d6864f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6874e6654d734910b67db92475ece16e', 'request_id': 'req-e0b38b7d-5c23-463b-82e4-ffef27d6864f'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:52.160637 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-e0b38b7d-5c23-463b-82e4-ffef27d6864f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6874e6654d734910b67db92475ece16e', 'request_id': 'req-e0b38b7d-5c23-463b-82e4-ffef27d6864f'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:52.171292 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-e0b38b7d-5c23-463b-82e4-ffef27d6864f admin admin] The token cache is being invalidate because project 6874e6654d734910b67db92475ece16e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:10:52.171588 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-e0b38b7d-5c23-463b-82e4-ffef27d6864f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:52.172236 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-e0b38b7d-5c23-463b-82e4-ffef27d6864f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:10:52.173790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1401/2806] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:10:51 2022] DELETE /identity/v3/projects/6874e6654d734910b67db92475ece16e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:10:59.068297 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a21fd-e90e-432f-b28c-3ffe2ba6dc09 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:10:59.069899 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-498a21fd-e90e-432f-b28c-3ffe2ba6dc09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:10:59.132604 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a21fd-e90e-432f-b28c-3ffe2ba6dc09 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:10:59.185780 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286f6b5-5df8-4b73-ab2c-5fafb05c33d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:10:59.186422 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286f6b5-5df8-4b73-ab2c-5fafb05c33d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:10:59.186705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286f6b5-5df8-4b73-ab2c-5fafb05c33d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:10:59.186981 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286f6b5-5df8-4b73-ab2c-5fafb05c33d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:10:59.189454 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6286f6b5-5df8-4b73-ab2c-5fafb05c33d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:10:59.200956 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6286f6b5-5df8-4b73-ab2c-5fafb05c33d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:10:59.202608 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6286f6b5-5df8-4b73-ab2c-5fafb05c33d9 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:10:59.213034 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1406/2807] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:10:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:11:02.168611 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=14,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:11:09.208651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=8,keystone:SELECT=66,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:11:10.320480 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b38b7d-5c23-463b-82e4-ffef27d6864f admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:11:10.322316 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-e0b38b7d-5c23-463b-82e4-ffef27d6864f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:11:10.407101 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b38b7d-5c23-463b-82e4-ffef27d6864f admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:11:10.413668 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023d57e-a39c-4292-a1ca-649e5cebfd50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:11:10.414335 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023d57e-a39c-4292-a1ca-649e5cebfd50 service nova] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:11:10.414614 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023d57e-a39c-4292-a1ca-649e5cebfd50 service nova] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:11:10.414888 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023d57e-a39c-4292-a1ca-649e5cebfd50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:11:10.416729 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-2023d57e-a39c-4292-a1ca-649e5cebfd50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:11:10.428963 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2023d57e-a39c-4292-a1ca-649e5cebfd50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:11:10.430567 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2023d57e-a39c-4292-a1ca-649e5cebfd50 service nova] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:11:10.441722 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1402/2808] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:11:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:11:20.437360 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=34 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:11:47.655511 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286f6b5-5df8-4b73-ab2c-5fafb05c33d9 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:11:47.687217 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286f6b5-5df8-4b73-ab2c-5fafb05c33d9 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:11:47.708892 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34163976-76e8-4ce7-9c55-dc4bf437e8c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:11:47.709558 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34163976-76e8-4ce7-9c55-dc4bf437e8c0 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:11:47.709838 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34163976-76e8-4ce7-9c55-dc4bf437e8c0 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:11:47.710131 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34163976-76e8-4ce7-9c55-dc4bf437e8c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:11:47.712004 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-34163976-76e8-4ce7-9c55-dc4bf437e8c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:11:47.727267 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34163976-76e8-4ce7-9c55-dc4bf437e8c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:11:47.728886 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34163976-76e8-4ce7-9c55-dc4bf437e8c0 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:11:47.753067 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1407/2809] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:11:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:11:56.360333 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023d57e-a39c-4292-a1ca-649e5cebfd50 service nova] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:11:56.361875 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-2023d57e-a39c-4292-a1ca-649e5cebfd50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:11:56.426262 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023d57e-a39c-4292-a1ca-649e5cebfd50 service nova] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:11:56.431229 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e2d82-1511-4c2a-8678-f3ca84ba333c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:11:56.431948 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e2d82-1511-4c2a-8678-f3ca84ba333c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:11:56.432208 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e2d82-1511-4c2a-8678-f3ca84ba333c admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:11:56.432470 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e2d82-1511-4c2a-8678-f3ca84ba333c admin admin] PATH_INFO: `/v3/users/92b9d212b86348c6b51a6b02818a0e80` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:11:56.433731 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6e2d82-1511-4c2a-8678-f3ca84ba333c admin admin] RBAC: Authorizing `identity:delete_user(user_id=92b9d212b86348c6b51a6b02818a0e80)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:11:56.436064 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6e2d82-1511-4c2a-8678-f3ca84ba333c admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:11:56.482197 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-ca6e2d82-1511-4c2a-8678-f3ca84ba333c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92b9d212b86348c6b51a6b02818a0e80'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:11:56.489122 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-ca6e2d82-1511-4c2a-8678-f3ca84ba333c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92b9d212b86348c6b51a6b02818a0e80'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:11:56.498119 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-ca6e2d82-1511-4c2a-8678-f3ca84ba333c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92b9d212b86348c6b51a6b02818a0e80'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:11:56.498705 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-ca6e2d82-1511-4c2a-8678-f3ca84ba333c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92b9d212b86348c6b51a6b02818a0e80'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:11:56.499378 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-ca6e2d82-1511-4c2a-8678-f3ca84ba333c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92b9d212b86348c6b51a6b02818a0e80'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:11:56.517773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1403/2810] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:11:56 2022] DELETE /identity/v3/users/92b9d212b86348c6b51a6b02818a0e80 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:11:56.699637 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34163976-76e8-4ce7-9c55-dc4bf437e8c0 service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:11:56.710830 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34163976-76e8-4ce7-9c55-dc4bf437e8c0 service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:11:56.780039 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9f217-397d-4c22-90c3-7fe09bf582b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:11:56.780693 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9f217-397d-4c22-90c3-7fe09bf582b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:11:56.780965 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9f217-397d-4c22-90c3-7fe09bf582b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:11:56.781243 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9f217-397d-4c22-90c3-7fe09bf582b1 admin admin] PATH_INFO: `/v3/projects/fd4481cb5d2f43c5b60db983132a3e16` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:11:56.782476 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c9f217-397d-4c22-90c3-7fe09bf582b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd4481cb5d2f43c5b60db983132a3e16)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:11:56.784534 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c9f217-397d-4c22-90c3-7fe09bf582b1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:11:56.832529 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-03c9f217-397d-4c22-90c3-7fe09bf582b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd4481cb5d2f43c5b60db983132a3e16', 'request_id': 'req-03c9f217-397d-4c22-90c3-7fe09bf582b1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:11:56.838955 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-03c9f217-397d-4c22-90c3-7fe09bf582b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd4481cb5d2f43c5b60db983132a3e16', 'request_id': 'req-03c9f217-397d-4c22-90c3-7fe09bf582b1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:11:56.846982 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-03c9f217-397d-4c22-90c3-7fe09bf582b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd4481cb5d2f43c5b60db983132a3e16', 'request_id': 'req-03c9f217-397d-4c22-90c3-7fe09bf582b1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:11:56.857482 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-03c9f217-397d-4c22-90c3-7fe09bf582b1 admin admin] The token cache is being invalidate because project fd4481cb5d2f43c5b60db983132a3e16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:11:56.857769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-03c9f217-397d-4c22-90c3-7fe09bf582b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:11:56.858406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-03c9f217-397d-4c22-90c3-7fe09bf582b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:11:56.859962 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1408/2811] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:11:56 2022] DELETE /identity/v3/projects/fd4481cb5d2f43c5b60db983132a3e16 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:11:56.865034 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc92c38-c897-416f-9287-75a1014f5bdf None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:11:56.865324 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc92c38-c897-416f-9287-75a1014f5bdf None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:11:56.865605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc92c38-c897-416f-9287-75a1014f5bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:11:56.926114 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-7cc92c38-c897-416f-9287-75a1014f5bdf None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:11:56.927507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-7cc92c38-c897-416f-9287-75a1014f5bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:11:56.977268 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:11:56.997635 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1404/2812] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:11:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:11:57.005609 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975e95d-70c3-4f51-9947-9fadb4310c05 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:11:57.005898 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975e95d-70c3-4f51-9947-9fadb4310c05 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:11:57.006174 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975e95d-70c3-4f51-9947-9fadb4310c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:11:57.026305 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5975e95d-70c3-4f51-9947-9fadb4310c05 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:11:57.027632 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5975e95d-70c3-4f51-9947-9fadb4310c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:11:57.036466 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1409/2813] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:11:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:11:57.041152 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc92c38-c897-416f-9287-75a1014f5bdf None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:11:57.057154 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc92c38-c897-416f-9287-75a1014f5bdf None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:11:57.064053 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa94e8-6cb3-4559-83ee-aa68397a6a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:11:57.064674 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa94e8-6cb3-4559-83ee-aa68397a6a7d admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:11:57.064957 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa94e8-6cb3-4559-83ee-aa68397a6a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:11:57.065238 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa94e8-6cb3-4559-83ee-aa68397a6a7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:11:57.065925 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa94e8-6cb3-4559-83ee-aa68397a6a7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:11:57.066241 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa94e8-6cb3-4559-83ee-aa68397a6a7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:11:57.067914 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa94e8-6cb3-4559-83ee-aa68397a6a7d admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:11:57.080411 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1405/2814] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:11:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:11:57.084759 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975e95d-70c3-4f51-9947-9fadb4310c05 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:11:57.094368 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975e95d-70c3-4f51-9947-9fadb4310c05 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:11:57.100390 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba97cd-36fa-4b72-a284-a28392a037ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:11:57.101026 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba97cd-36fa-4b72-a284-a28392a037ac admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:11:57.101310 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba97cd-36fa-4b72-a284-a28392a037ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:11:57.101550 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba97cd-36fa-4b72-a284-a28392a037ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:11:57.102238 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba97cd-36fa-4b72-a284-a28392a037ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:11:57.104180 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba97cd-36fa-4b72-a284-a28392a037ac admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:11:57.144254 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1410/2815] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:11:57 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:11:57.149074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa94e8-6cb3-4559-83ee-aa68397a6a7d admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:11:57.160836 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa94e8-6cb3-4559-83ee-aa68397a6a7d admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:11:57.224711 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7687355a-9325-4fec-961a-f7f6e4fc600e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:11:57.225256 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687355a-9325-4fec-961a-f7f6e4fc600e admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:11:57.225540 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687355a-9325-4fec-961a-f7f6e4fc600e admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:11:57.225809 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687355a-9325-4fec-961a-f7f6e4fc600e admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:11:57.226475 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7687355a-9325-4fec-961a-f7f6e4fc600e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:11:57.228447 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7687355a-9325-4fec-961a-f7f6e4fc600e admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:11:57.251898 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1406/2816] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:11:57 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:11:57.256099 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba97cd-36fa-4b72-a284-a28392a037ac admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:11:57.266411 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba97cd-36fa-4b72-a284-a28392a037ac admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:11:57.271151 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30901c05-9ad8-4b0d-b59b-1dd1b5552b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:11:57.271789 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30901c05-9ad8-4b0d-b59b-1dd1b5552b55 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:11:57.272072 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30901c05-9ad8-4b0d-b59b-1dd1b5552b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:11:57.272344 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30901c05-9ad8-4b0d-b59b-1dd1b5552b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:11:57.273039 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30901c05-9ad8-4b0d-b59b-1dd1b5552b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:11:57.274639 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30901c05-9ad8-4b0d-b59b-1dd1b5552b55 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:11:57.284437 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1411/2817] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:11:57.288176 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7687355a-9325-4fec-961a-f7f6e4fc600e admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:11:57.299046 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7687355a-9325-4fec-961a-f7f6e4fc600e admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:11:57.304930 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e245dd-4177-4cc6-a073-dade9cd4b1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:11:57.305573 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e245dd-4177-4cc6-a073-dade9cd4b1b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:11:57.305858 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e245dd-4177-4cc6-a073-dade9cd4b1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:11:57.306135 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e245dd-4177-4cc6-a073-dade9cd4b1b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:11:57.306847 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e245dd-4177-4cc6-a073-dade9cd4b1b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:11:57.308595 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e245dd-4177-4cc6-a073-dade9cd4b1b7 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:11:57.319604 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1407/2818] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:11:57.324190 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30901c05-9ad8-4b0d-b59b-1dd1b5552b55 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:11:57.334627 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30901c05-9ad8-4b0d-b59b-1dd1b5552b55 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:11:57.340294 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c7a86-2a92-43d6-b058-7980bb591950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:11:57.340983 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c7a86-2a92-43d6-b058-7980bb591950 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:11:57.341272 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c7a86-2a92-43d6-b058-7980bb591950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:11:57.341552 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c7a86-2a92-43d6-b058-7980bb591950 admin admin] PATH_INFO: `/v3/projects/6413758618124b4c95d937e8bd7d2e4c/users/6b9ead798ac54906ad5cccc7a826daac/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:11:57.357478 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c7a86-2a92-43d6-b058-7980bb591950 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6413758618124b4c95d937e8bd7d2e4c, user_id=6b9ead798ac54906ad5cccc7a826daac, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:11:57.360191 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c7a86-2a92-43d6-b058-7980bb591950 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:11:57.377646 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1412/2819] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:11:57 2022] PUT /identity/v3/projects/6413758618124b4c95d937e8bd7d2e4c/users/6b9ead798ac54906ad5cccc7a826daac/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:11:57.388394 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e14f9-2358-4b44-868c-2ec70a39d237 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:11:57.388939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e14f9-2358-4b44-868c-2ec70a39d237 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:11:57.388939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e14f9-2358-4b44-868c-2ec70a39d237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:11:57.418381 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-b47e14f9-2358-4b44-868c-2ec70a39d237 None None] MFA Rules not processed for user `6b9ead798ac54906ad5cccc7a826daac`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:11:57.419583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-b47e14f9-2358-4b44-868c-2ec70a39d237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:11:57.495125 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1408/2820] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:11:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:11:57.511612 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c7a86-2a92-43d6-b058-7980bb591950 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:11:57.513703 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5f3c7a86-2a92-43d6-b058-7980bb591950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:11:57.521196 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:11:57.595607 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c7a86-2a92-43d6-b058-7980bb591950 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:11:57.601964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87534bd1-a9a5-4d41-814b-0ccd17c23098 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:11:57.602601 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87534bd1-a9a5-4d41-814b-0ccd17c23098 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:11:57.602881 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87534bd1-a9a5-4d41-814b-0ccd17c23098 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:11:57.603128 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87534bd1-a9a5-4d41-814b-0ccd17c23098 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:11:57.611884 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87534bd1-a9a5-4d41-814b-0ccd17c23098 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:11:57.613483 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87534bd1-a9a5-4d41-814b-0ccd17c23098 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:11:57.622609 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1413/2821] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:11:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:12:04.965586 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e14f9-2358-4b44-868c-2ec70a39d237 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:05.010716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:12:05.029048 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e14f9-2358-4b44-868c-2ec70a39d237 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:05.033822 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7323e6ab-ef20-4c7e-b15e-46a545208df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:05.034512 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323e6ab-ef20-4c7e-b15e-46a545208df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:05.034820 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323e6ab-ef20-4c7e-b15e-46a545208df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:05.035095 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323e6ab-ef20-4c7e-b15e-46a545208df8 admin admin] PATH_INFO: `/v3/users/6b9ead798ac54906ad5cccc7a826daac` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:05.036338 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7323e6ab-ef20-4c7e-b15e-46a545208df8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b9ead798ac54906ad5cccc7a826daac)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:05.038310 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7323e6ab-ef20-4c7e-b15e-46a545208df8 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:05.087065 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-7323e6ab-ef20-4c7e-b15e-46a545208df8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b9ead798ac54906ad5cccc7a826daac'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:05.094001 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-7323e6ab-ef20-4c7e-b15e-46a545208df8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b9ead798ac54906ad5cccc7a826daac'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:05.102605 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-7323e6ab-ef20-4c7e-b15e-46a545208df8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b9ead798ac54906ad5cccc7a826daac'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:05.103497 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-7323e6ab-ef20-4c7e-b15e-46a545208df8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b9ead798ac54906ad5cccc7a826daac'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:05.104142 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-7323e6ab-ef20-4c7e-b15e-46a545208df8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b9ead798ac54906ad5cccc7a826daac'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:05.124229 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1409/2822] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:12:04 2022] DELETE /identity/v3/users/6b9ead798ac54906ad5cccc7a826daac => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:12:05.128896 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43565d98-e028-4f15-a9e9-22567f8d5ce6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:05.129398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43565d98-e028-4f15-a9e9-22567f8d5ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:05.129505 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43565d98-e028-4f15-a9e9-22567f8d5ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:05.154643 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-43565d98-e028-4f15-a9e9-22567f8d5ce6 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:12:05.156038 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-43565d98-e028-4f15-a9e9-22567f8d5ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:05.217142 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1414/2823] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:12:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:12:05.227187 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7323e6ab-ef20-4c7e-b15e-46a545208df8 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:05.228679 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-7323e6ab-ef20-4c7e-b15e-46a545208df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:05.287572 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7323e6ab-ef20-4c7e-b15e-46a545208df8 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:05.291583 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f8455-be38-4f2e-aa07-e1a3178bc9d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:05.292243 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f8455-be38-4f2e-aa07-e1a3178bc9d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:05.292522 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f8455-be38-4f2e-aa07-e1a3178bc9d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:05.292803 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f8455-be38-4f2e-aa07-e1a3178bc9d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:05.302492 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082f8455-be38-4f2e-aa07-e1a3178bc9d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:05.304000 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082f8455-be38-4f2e-aa07-e1a3178bc9d9 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:05.314659 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1410/2824] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:12:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:12:05.599057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43565d98-e028-4f15-a9e9-22567f8d5ce6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:05.600663 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-43565d98-e028-4f15-a9e9-22567f8d5ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:05.615387 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43565d98-e028-4f15-a9e9-22567f8d5ce6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:05.620220 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e813d9-6138-452c-aa52-fc1b718e2f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:05.620875 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e813d9-6138-452c-aa52-fc1b718e2f08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:05.621155 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e813d9-6138-452c-aa52-fc1b718e2f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:05.621428 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e813d9-6138-452c-aa52-fc1b718e2f08 admin admin] PATH_INFO: `/v3/projects/6413758618124b4c95d937e8bd7d2e4c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:05.622604 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e813d9-6138-452c-aa52-fc1b718e2f08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6413758618124b4c95d937e8bd7d2e4c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:05.624596 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e813d9-6138-452c-aa52-fc1b718e2f08 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:05.675622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b3e813d9-6138-452c-aa52-fc1b718e2f08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6413758618124b4c95d937e8bd7d2e4c', 'request_id': 'req-b3e813d9-6138-452c-aa52-fc1b718e2f08'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:05.681959 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b3e813d9-6138-452c-aa52-fc1b718e2f08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6413758618124b4c95d937e8bd7d2e4c', 'request_id': 'req-b3e813d9-6138-452c-aa52-fc1b718e2f08'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:05.691438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b3e813d9-6138-452c-aa52-fc1b718e2f08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6413758618124b4c95d937e8bd7d2e4c', 'request_id': 'req-b3e813d9-6138-452c-aa52-fc1b718e2f08'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:05.702426 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b3e813d9-6138-452c-aa52-fc1b718e2f08 admin admin] The token cache is being invalidate because project 6413758618124b4c95d937e8bd7d2e4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:12:05.702719 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b3e813d9-6138-452c-aa52-fc1b718e2f08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:05.703377 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b3e813d9-6138-452c-aa52-fc1b718e2f08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:05.704958 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1415/2825] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:12:05 2022] DELETE /identity/v3/projects/6413758618124b4c95d937e8bd7d2e4c => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:12:05.709987 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dd008-0266-4b99-87ea-48e9aee63d90 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:05.710320 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dd008-0266-4b99-87ea-48e9aee63d90 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:05.710540 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dd008-0266-4b99-87ea-48e9aee63d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:05.730897 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-c40dd008-0266-4b99-87ea-48e9aee63d90 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:12:05.732140 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-c40dd008-0266-4b99-87ea-48e9aee63d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:05.793895 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1411/2826] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:12:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:12:05.803389 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1651538-33cc-4364-8833-82c05a20b908 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:05.803742 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1651538-33cc-4364-8833-82c05a20b908 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:05.804022 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1651538-33cc-4364-8833-82c05a20b908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:05.825564 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d1651538-33cc-4364-8833-82c05a20b908 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:12:05.826809 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d1651538-33cc-4364-8833-82c05a20b908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:05.836310 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1416/2827] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:12:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:12:05.840725 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40dd008-0266-4b99-87ea-48e9aee63d90 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:05.851342 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40dd008-0266-4b99-87ea-48e9aee63d90 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:05.856954 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70adb887-bd99-4718-9949-529af7b61036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:05.857578 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70adb887-bd99-4718-9949-529af7b61036 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:05.857893 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70adb887-bd99-4718-9949-529af7b61036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:05.858169 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70adb887-bd99-4718-9949-529af7b61036 admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:05.858880 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70adb887-bd99-4718-9949-529af7b61036 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:12:05.859185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70adb887-bd99-4718-9949-529af7b61036 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:05.860861 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70adb887-bd99-4718-9949-529af7b61036 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:05.862369 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:12:05.874292 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1412/2828] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:12:05.878866 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1651538-33cc-4364-8833-82c05a20b908 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:05.888306 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1651538-33cc-4364-8833-82c05a20b908 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:05.893259 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b968d-6095-4538-af89-6896d397eb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:05.893875 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b968d-6095-4538-af89-6896d397eb03 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:05.894163 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b968d-6095-4538-af89-6896d397eb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:05.894438 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b968d-6095-4538-af89-6896d397eb03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:05.895103 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5b968d-6095-4538-af89-6896d397eb03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:05.897067 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5b968d-6095-4538-af89-6896d397eb03 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:05.901612 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:12:05.928135 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1417/2829] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:12:05 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:12:05.932981 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70adb887-bd99-4718-9949-529af7b61036 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:05.944346 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70adb887-bd99-4718-9949-529af7b61036 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:06.002546 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266bec0-1706-4030-a773-9cbb8a335865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:06.003185 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266bec0-1706-4030-a773-9cbb8a335865 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:06.003441 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266bec0-1706-4030-a773-9cbb8a335865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:06.003745 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266bec0-1706-4030-a773-9cbb8a335865 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:06.004410 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266bec0-1706-4030-a773-9cbb8a335865 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:06.006354 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266bec0-1706-4030-a773-9cbb8a335865 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:06.032000 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1413/2830] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:12:05 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:12:06.036310 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b968d-6095-4538-af89-6896d397eb03 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:06.047709 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b968d-6095-4538-af89-6896d397eb03 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:06.053441 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08246c3-01f7-4e20-aba2-82319889e819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:06.054066 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08246c3-01f7-4e20-aba2-82319889e819 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:06.054350 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08246c3-01f7-4e20-aba2-82319889e819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:06.054622 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08246c3-01f7-4e20-aba2-82319889e819 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:06.055325 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08246c3-01f7-4e20-aba2-82319889e819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:06.056988 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08246c3-01f7-4e20-aba2-82319889e819 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:06.067941 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1418/2831] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:12:06.072007 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266bec0-1706-4030-a773-9cbb8a335865 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:06.082471 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266bec0-1706-4030-a773-9cbb8a335865 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:06.087639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4034db9-8529-4279-9267-8fe4553146da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:06.088253 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4034db9-8529-4279-9267-8fe4553146da admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:06.088538 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4034db9-8529-4279-9267-8fe4553146da admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:06.088815 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4034db9-8529-4279-9267-8fe4553146da admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:06.089518 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4034db9-8529-4279-9267-8fe4553146da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:06.091132 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4034db9-8529-4279-9267-8fe4553146da admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:06.102276 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1414/2832] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:12:06.106544 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08246c3-01f7-4e20-aba2-82319889e819 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:06.117055 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08246c3-01f7-4e20-aba2-82319889e819 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:06.122921 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d57633-b029-4037-b5b6-ecc2d1377dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:06.123625 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d57633-b029-4037-b5b6-ecc2d1377dea admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:06.124003 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d57633-b029-4037-b5b6-ecc2d1377dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:06.124277 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d57633-b029-4037-b5b6-ecc2d1377dea admin admin] PATH_INFO: `/v3/projects/a3c5cdfc4b3f4efaaf488ee49509ba95/users/8cea8ce3cc7940dfb21f197b8886ca82/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:06.139733 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d57633-b029-4037-b5b6-ecc2d1377dea admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3c5cdfc4b3f4efaaf488ee49509ba95, user_id=8cea8ce3cc7940dfb21f197b8886ca82, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:06.142373 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d57633-b029-4037-b5b6-ecc2d1377dea admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:06.157328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1419/2833] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:12:06 2022] PUT /identity/v3/projects/a3c5cdfc4b3f4efaaf488ee49509ba95/users/8cea8ce3cc7940dfb21f197b8886ca82/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:12:06.177511 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e52b9c-e73c-4549-8c5b-15ff9f9d7717 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:06.177799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e52b9c-e73c-4549-8c5b-15ff9f9d7717 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:06.178074 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e52b9c-e73c-4549-8c5b-15ff9f9d7717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:06.207911 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-d2e52b9c-e73c-4549-8c5b-15ff9f9d7717 None None] MFA Rules not processed for user `8cea8ce3cc7940dfb21f197b8886ca82`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:12:06.209424 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-d2e52b9c-e73c-4549-8c5b-15ff9f9d7717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:06.273532 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1415/2834] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:12:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:12:06.288939 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d57633-b029-4037-b5b6-ecc2d1377dea admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:06.291669 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-97d57633-b029-4037-b5b6-ecc2d1377dea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:06.384647 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d57633-b029-4037-b5b6-ecc2d1377dea admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:06.388859 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c4a4d-5210-408c-8211-f644602986a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:06.389582 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c4a4d-5210-408c-8211-f644602986a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:06.389867 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c4a4d-5210-408c-8211-f644602986a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:06.390127 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c4a4d-5210-408c-8211-f644602986a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:06.399347 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c4a4d-5210-408c-8211-f644602986a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:06.400829 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c4a4d-5210-408c-8211-f644602986a0 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:06.409884 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1420/2835] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:12:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:12:13.874742 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e52b9c-e73c-4549-8c5b-15ff9f9d7717 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:13.944852 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e52b9c-e73c-4549-8c5b-15ff9f9d7717 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:13.950168 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4084842-c504-45e7-9037-ba6a86556965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:13.950808 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4084842-c504-45e7-9037-ba6a86556965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:13.951092 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4084842-c504-45e7-9037-ba6a86556965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:13.951375 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4084842-c504-45e7-9037-ba6a86556965 admin admin] PATH_INFO: `/v3/users/8cea8ce3cc7940dfb21f197b8886ca82` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:13.952696 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4084842-c504-45e7-9037-ba6a86556965 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cea8ce3cc7940dfb21f197b8886ca82)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:13.954716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4084842-c504-45e7-9037-ba6a86556965 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:13.957331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:12:14.005375 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-b4084842-c504-45e7-9037-ba6a86556965 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cea8ce3cc7940dfb21f197b8886ca82'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:14.012288 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-b4084842-c504-45e7-9037-ba6a86556965 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cea8ce3cc7940dfb21f197b8886ca82'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:14.021155 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-b4084842-c504-45e7-9037-ba6a86556965 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cea8ce3cc7940dfb21f197b8886ca82'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:14.021794 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-b4084842-c504-45e7-9037-ba6a86556965 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cea8ce3cc7940dfb21f197b8886ca82'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:14.022458 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-b4084842-c504-45e7-9037-ba6a86556965 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cea8ce3cc7940dfb21f197b8886ca82'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:14.039868 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1416/2836] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:12:13 2022] DELETE /identity/v3/users/8cea8ce3cc7940dfb21f197b8886ca82 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:12:14.044472 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d556e05-0b84-4870-88a5-8a84fbb5e83e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:14.044968 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d556e05-0b84-4870-88a5-8a84fbb5e83e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:14.045069 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d556e05-0b84-4870-88a5-8a84fbb5e83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:14.066874 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7d556e05-0b84-4870-88a5-8a84fbb5e83e None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:12:14.068146 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7d556e05-0b84-4870-88a5-8a84fbb5e83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:14.135979 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1421/2837] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:12:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:12:14.146271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4084842-c504-45e7-9037-ba6a86556965 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:14.148125 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-b4084842-c504-45e7-9037-ba6a86556965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:14.208802 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4084842-c504-45e7-9037-ba6a86556965 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:14.214481 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ff3ed-2392-47e8-8eb6-1ba09d93d3d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:14.215395 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ff3ed-2392-47e8-8eb6-1ba09d93d3d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:14.215702 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ff3ed-2392-47e8-8eb6-1ba09d93d3d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:14.216003 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ff3ed-2392-47e8-8eb6-1ba09d93d3d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:14.226980 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ff3ed-2392-47e8-8eb6-1ba09d93d3d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:14.228468 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ff3ed-2392-47e8-8eb6-1ba09d93d3d7 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:14.238278 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1417/2838] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:12:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:12:14.524902 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d556e05-0b84-4870-88a5-8a84fbb5e83e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:14.526344 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7d556e05-0b84-4870-88a5-8a84fbb5e83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:14.538887 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d556e05-0b84-4870-88a5-8a84fbb5e83e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:14.544524 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e115896-7148-4ae9-abfc-6795b4b39fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:14.545183 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e115896-7148-4ae9-abfc-6795b4b39fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:14.545465 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e115896-7148-4ae9-abfc-6795b4b39fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:14.545773 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e115896-7148-4ae9-abfc-6795b4b39fd5 admin admin] PATH_INFO: `/v3/projects/a3c5cdfc4b3f4efaaf488ee49509ba95` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:14.547057 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e115896-7148-4ae9-abfc-6795b4b39fd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3c5cdfc4b3f4efaaf488ee49509ba95)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:14.549213 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e115896-7148-4ae9-abfc-6795b4b39fd5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:14.583409 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:12:14.600680 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5e115896-7148-4ae9-abfc-6795b4b39fd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3c5cdfc4b3f4efaaf488ee49509ba95', 'request_id': 'req-5e115896-7148-4ae9-abfc-6795b4b39fd5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:14.606749 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5e115896-7148-4ae9-abfc-6795b4b39fd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3c5cdfc4b3f4efaaf488ee49509ba95', 'request_id': 'req-5e115896-7148-4ae9-abfc-6795b4b39fd5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:14.615969 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5e115896-7148-4ae9-abfc-6795b4b39fd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3c5cdfc4b3f4efaaf488ee49509ba95', 'request_id': 'req-5e115896-7148-4ae9-abfc-6795b4b39fd5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:14.627157 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5e115896-7148-4ae9-abfc-6795b4b39fd5 admin admin] The token cache is being invalidate because project a3c5cdfc4b3f4efaaf488ee49509ba95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:12:14.627448 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5e115896-7148-4ae9-abfc-6795b4b39fd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:14.628119 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5e115896-7148-4ae9-abfc-6795b4b39fd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:14.629665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1422/2839] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:12:14 2022] DELETE /identity/v3/projects/a3c5cdfc4b3f4efaaf488ee49509ba95 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:12:14.634684 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4a022-4000-4f38-b19d-628073736e8f None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:14.634970 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4a022-4000-4f38-b19d-628073736e8f None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:14.635243 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4a022-4000-4f38-b19d-628073736e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:14.656127 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-8ac4a022-4000-4f38-b19d-628073736e8f None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:12:14.657373 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-8ac4a022-4000-4f38-b19d-628073736e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:14.717952 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1418/2840] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:12:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:12:14.726296 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339711e-fdf2-4977-bb31-b73d91285ebd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:14.726592 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339711e-fdf2-4977-bb31-b73d91285ebd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:14.726872 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339711e-fdf2-4977-bb31-b73d91285ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:14.747383 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f339711e-fdf2-4977-bb31-b73d91285ebd None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:12:14.748626 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f339711e-fdf2-4977-bb31-b73d91285ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:14.757402 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1423/2841] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:12:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:12:14.763382 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac4a022-4000-4f38-b19d-628073736e8f None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:14.775842 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac4a022-4000-4f38-b19d-628073736e8f None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:14.781949 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddad74e-5b6f-4543-b3fe-f20953019c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:14.782573 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddad74e-5b6f-4543-b3fe-f20953019c8e admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:14.782828 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddad74e-5b6f-4543-b3fe-f20953019c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:14.783099 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddad74e-5b6f-4543-b3fe-f20953019c8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:14.783985 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddad74e-5b6f-4543-b3fe-f20953019c8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:12:14.784290 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddad74e-5b6f-4543-b3fe-f20953019c8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:14.785910 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddad74e-5b6f-4543-b3fe-f20953019c8e admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:14.798370 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1419/2842] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:12:14.802735 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339711e-fdf2-4977-bb31-b73d91285ebd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:14.812417 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339711e-fdf2-4977-bb31-b73d91285ebd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:14.817370 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb24b8a-3d1d-4edc-bb69-7bdc5da2ed1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:14.817990 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb24b8a-3d1d-4edc-bb69-7bdc5da2ed1d admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:14.818274 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb24b8a-3d1d-4edc-bb69-7bdc5da2ed1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:14.818552 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb24b8a-3d1d-4edc-bb69-7bdc5da2ed1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:14.819216 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb24b8a-3d1d-4edc-bb69-7bdc5da2ed1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:14.821230 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb24b8a-3d1d-4edc-bb69-7bdc5da2ed1d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:14.851299 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1424/2843] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:12:14 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:12:14.855790 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddad74e-5b6f-4543-b3fe-f20953019c8e admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:14.866322 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddad74e-5b6f-4543-b3fe-f20953019c8e admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:14.874934 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:12:14.925581 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acffaa01-b7ec-40d0-a172-afcd568f6c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:14.926203 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acffaa01-b7ec-40d0-a172-afcd568f6c84 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:14.926488 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acffaa01-b7ec-40d0-a172-afcd568f6c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:14.926759 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acffaa01-b7ec-40d0-a172-afcd568f6c84 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:14.927436 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acffaa01-b7ec-40d0-a172-afcd568f6c84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:14.929401 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acffaa01-b7ec-40d0-a172-afcd568f6c84 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:14.953650 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1420/2844] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:12:14 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:12:14.958536 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb24b8a-3d1d-4edc-bb69-7bdc5da2ed1d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:14.973096 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb24b8a-3d1d-4edc-bb69-7bdc5da2ed1d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:14.977976 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591bd69-675f-4fed-8a6f-bd39412c8ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:14.978595 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591bd69-675f-4fed-8a6f-bd39412c8ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:14.978879 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591bd69-675f-4fed-8a6f-bd39412c8ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:14.979183 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591bd69-675f-4fed-8a6f-bd39412c8ce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:14.979910 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591bd69-675f-4fed-8a6f-bd39412c8ce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:14.981524 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591bd69-675f-4fed-8a6f-bd39412c8ce5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:14.991969 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1425/2845] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 05:12:14.995788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acffaa01-b7ec-40d0-a172-afcd568f6c84 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:15.005988 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acffaa01-b7ec-40d0-a172-afcd568f6c84 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:15.011727 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efca20d-f71b-474e-8fa7-c5dced606203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:15.012348 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efca20d-f71b-474e-8fa7-c5dced606203 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:15.012632 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efca20d-f71b-474e-8fa7-c5dced606203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:15.012913 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efca20d-f71b-474e-8fa7-c5dced606203 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:15.013608 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efca20d-f71b-474e-8fa7-c5dced606203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:15.015229 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efca20d-f71b-474e-8fa7-c5dced606203 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:15.026350 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1421/2846] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:12:15.030554 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591bd69-675f-4fed-8a6f-bd39412c8ce5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:15.040602 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591bd69-675f-4fed-8a6f-bd39412c8ce5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:15.045686 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8a3b2-7cfc-4ecf-a383-9c11f266f79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:15.046347 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8a3b2-7cfc-4ecf-a383-9c11f266f79f admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:15.046632 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8a3b2-7cfc-4ecf-a383-9c11f266f79f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:15.046904 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8a3b2-7cfc-4ecf-a383-9c11f266f79f admin admin] PATH_INFO: `/v3/projects/349f608783d24fd4a67769a26ff41fc9/users/f69603289c5346cd8ffc5e7e15e34181/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:15.062269 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc8a3b2-7cfc-4ecf-a383-9c11f266f79f admin admin] RBAC: Authorizing `identity:create_grant(project_id=349f608783d24fd4a67769a26ff41fc9, user_id=f69603289c5346cd8ffc5e7e15e34181, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:15.065016 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc8a3b2-7cfc-4ecf-a383-9c11f266f79f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:15.079599 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1426/2847] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:12:15 2022] PUT /identity/v3/projects/349f608783d24fd4a67769a26ff41fc9/users/f69603289c5346cd8ffc5e7e15e34181/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:12:15.151746 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77ae92-9913-45f0-b1bc-d02ee9f168e6 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:15.152048 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77ae92-9913-45f0-b1bc-d02ee9f168e6 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:15.152425 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77ae92-9913-45f0-b1bc-d02ee9f168e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:15.182277 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-1a77ae92-9913-45f0-b1bc-d02ee9f168e6 None None] MFA Rules not processed for user `f69603289c5346cd8ffc5e7e15e34181`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:12:15.183508 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-1a77ae92-9913-45f0-b1bc-d02ee9f168e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:15.249755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1422/2848] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:12:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:12:15.261141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8a3b2-7cfc-4ecf-a383-9c11f266f79f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:15.263302 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dcc8a3b2-7cfc-4ecf-a383-9c11f266f79f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:15.323869 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8a3b2-7cfc-4ecf-a383-9c11f266f79f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:15.328113 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53adc64-4cc8-4c49-84bd-5dacb5d6052d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:15.328831 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53adc64-4cc8-4c49-84bd-5dacb5d6052d service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:15.329120 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53adc64-4cc8-4c49-84bd-5dacb5d6052d service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:15.329398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53adc64-4cc8-4c49-84bd-5dacb5d6052d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:15.338307 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53adc64-4cc8-4c49-84bd-5dacb5d6052d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:15.339787 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53adc64-4cc8-4c49-84bd-5dacb5d6052d service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:15.348812 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1427/2849] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:12:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:12:18.418325 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77ae92-9913-45f0-b1bc-d02ee9f168e6 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:18.483437 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77ae92-9913-45f0-b1bc-d02ee9f168e6 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:18.488680 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde4f422-e40b-46a4-afb4-e28e07f9c7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:18.489322 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde4f422-e40b-46a4-afb4-e28e07f9c7ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:18.489594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde4f422-e40b-46a4-afb4-e28e07f9c7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:18.489875 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde4f422-e40b-46a4-afb4-e28e07f9c7ff admin admin] PATH_INFO: `/v3/users/f69603289c5346cd8ffc5e7e15e34181` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:18.491127 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde4f422-e40b-46a4-afb4-e28e07f9c7ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=f69603289c5346cd8ffc5e7e15e34181)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:18.493108 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde4f422-e40b-46a4-afb4-e28e07f9c7ff admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:18.513984 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:12:18.540462 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-dde4f422-e40b-46a4-afb4-e28e07f9c7ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f69603289c5346cd8ffc5e7e15e34181'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:18.547801 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-dde4f422-e40b-46a4-afb4-e28e07f9c7ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f69603289c5346cd8ffc5e7e15e34181'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:18.557452 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-dde4f422-e40b-46a4-afb4-e28e07f9c7ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f69603289c5346cd8ffc5e7e15e34181'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:18.558335 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-dde4f422-e40b-46a4-afb4-e28e07f9c7ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f69603289c5346cd8ffc5e7e15e34181'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:18.559419 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-dde4f422-e40b-46a4-afb4-e28e07f9c7ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f69603289c5346cd8ffc5e7e15e34181'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:18.577309 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1423/2850] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:12:18 2022] DELETE /identity/v3/users/f69603289c5346cd8ffc5e7e15e34181 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:12:18.581978 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712a792-00d7-4b5e-83d7-c743fd2ff809 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:18.582464 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712a792-00d7-4b5e-83d7-c743fd2ff809 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:18.582713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712a792-00d7-4b5e-83d7-c743fd2ff809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:18.603594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d712a792-00d7-4b5e-83d7-c743fd2ff809 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:12:18.604839 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d712a792-00d7-4b5e-83d7-c743fd2ff809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:18.654871 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:12:18.674571 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1428/2851] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:12:18.682244 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde4f422-e40b-46a4-afb4-e28e07f9c7ff admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:18.683777 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-dde4f422-e40b-46a4-afb4-e28e07f9c7ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:18.740846 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde4f422-e40b-46a4-afb4-e28e07f9c7ff admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:18.745312 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc99b4-084f-4a9f-a0d4-9c222efb9d2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:18.745931 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc99b4-084f-4a9f-a0d4-9c222efb9d2c service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:18.746210 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc99b4-084f-4a9f-a0d4-9c222efb9d2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:18.746490 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc99b4-084f-4a9f-a0d4-9c222efb9d2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:18.756153 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fc99b4-084f-4a9f-a0d4-9c222efb9d2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:18.757594 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fc99b4-084f-4a9f-a0d4-9c222efb9d2c service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:18.767024 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1424/2852] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:12:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:12:19.034613 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d712a792-00d7-4b5e-83d7-c743fd2ff809 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:19.036195 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d712a792-00d7-4b5e-83d7-c743fd2ff809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:19.048311 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d712a792-00d7-4b5e-83d7-c743fd2ff809 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:19.054495 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa44242-e66f-4a3e-90ca-e9c5cf6d5cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:19.054849 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa44242-e66f-4a3e-90ca-e9c5cf6d5cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:19.055139 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa44242-e66f-4a3e-90ca-e9c5cf6d5cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:19.055406 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa44242-e66f-4a3e-90ca-e9c5cf6d5cae admin admin] PATH_INFO: `/v3/projects/349f608783d24fd4a67769a26ff41fc9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:19.056713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa44242-e66f-4a3e-90ca-e9c5cf6d5cae admin admin] RBAC: Authorizing `identity:delete_project(project_id=349f608783d24fd4a67769a26ff41fc9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:19.059491 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa44242-e66f-4a3e-90ca-e9c5cf6d5cae admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:19.108086 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2fa44242-e66f-4a3e-90ca-e9c5cf6d5cae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '349f608783d24fd4a67769a26ff41fc9', 'request_id': 'req-2fa44242-e66f-4a3e-90ca-e9c5cf6d5cae'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:19.114938 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2fa44242-e66f-4a3e-90ca-e9c5cf6d5cae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '349f608783d24fd4a67769a26ff41fc9', 'request_id': 'req-2fa44242-e66f-4a3e-90ca-e9c5cf6d5cae'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:19.125650 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2fa44242-e66f-4a3e-90ca-e9c5cf6d5cae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '349f608783d24fd4a67769a26ff41fc9', 'request_id': 'req-2fa44242-e66f-4a3e-90ca-e9c5cf6d5cae'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:19.136922 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2fa44242-e66f-4a3e-90ca-e9c5cf6d5cae admin admin] The token cache is being invalidate because project 349f608783d24fd4a67769a26ff41fc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:12:19.137205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2fa44242-e66f-4a3e-90ca-e9c5cf6d5cae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:19.137886 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2fa44242-e66f-4a3e-90ca-e9c5cf6d5cae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:19.139410 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1429/2853] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:12:19 2022] DELETE /identity/v3/projects/349f608783d24fd4a67769a26ff41fc9 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:12:19.145481 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4cb09-1e80-4674-832e-3cf34a10cf2d None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:19.145768 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4cb09-1e80-4674-832e-3cf34a10cf2d None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:19.146042 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4cb09-1e80-4674-832e-3cf34a10cf2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:19.167526 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-7ea4cb09-1e80-4674-832e-3cf34a10cf2d None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:12:19.168753 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-7ea4cb09-1e80-4674-832e-3cf34a10cf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:19.230576 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1425/2854] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:12:19.238705 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71592aad-628c-499f-97a0-6e25f5d94f28 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:19.238990 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71592aad-628c-499f-97a0-6e25f5d94f28 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:19.239292 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71592aad-628c-499f-97a0-6e25f5d94f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:19.259360 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-71592aad-628c-499f-97a0-6e25f5d94f28 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:12:19.260614 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-71592aad-628c-499f-97a0-6e25f5d94f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:19.269131 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1430/2855] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:12:19.273644 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4cb09-1e80-4674-832e-3cf34a10cf2d None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:19.283820 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4cb09-1e80-4674-832e-3cf34a10cf2d None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:19.289650 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d182a92-349e-4a09-91c7-ce553ece30b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:19.290268 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d182a92-349e-4a09-91c7-ce553ece30b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:19.290554 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d182a92-349e-4a09-91c7-ce553ece30b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:19.290828 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d182a92-349e-4a09-91c7-ce553ece30b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:19.291530 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d182a92-349e-4a09-91c7-ce553ece30b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:12:19.291872 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d182a92-349e-4a09-91c7-ce553ece30b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:19.293480 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d182a92-349e-4a09-91c7-ce553ece30b0 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:19.305415 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1426/2856] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:12:19.310486 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71592aad-628c-499f-97a0-6e25f5d94f28 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:19.319985 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71592aad-628c-499f-97a0-6e25f5d94f28 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:19.325140 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5ab49-6a75-44cf-b6ed-0a792c53c179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:19.325754 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5ab49-6a75-44cf-b6ed-0a792c53c179 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:19.326036 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5ab49-6a75-44cf-b6ed-0a792c53c179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:19.326310 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5ab49-6a75-44cf-b6ed-0a792c53c179 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:19.327017 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5ab49-6a75-44cf-b6ed-0a792c53c179 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:19.328950 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5ab49-6a75-44cf-b6ed-0a792c53c179 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:19.358737 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1431/2857] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:12:19 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:12:19.363238 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d182a92-349e-4a09-91c7-ce553ece30b0 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:19.373307 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d182a92-349e-4a09-91c7-ce553ece30b0 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:19.401507 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:12:19.435982 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7893ff-3118-4d50-9c21-9861583a75a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:19.436636 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7893ff-3118-4d50-9c21-9861583a75a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:19.436823 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7893ff-3118-4d50-9c21-9861583a75a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:19.437104 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7893ff-3118-4d50-9c21-9861583a75a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:19.437785 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7893ff-3118-4d50-9c21-9861583a75a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:19.439939 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7893ff-3118-4d50-9c21-9861583a75a0 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:19.464009 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1427/2858] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:12:19 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:12:19.468240 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5ab49-6a75-44cf-b6ed-0a792c53c179 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:19.478784 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5ab49-6a75-44cf-b6ed-0a792c53c179 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:19.483592 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3cff7-5d08-47f0-8a85-5c69c1f752db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:19.484234 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3cff7-5d08-47f0-8a85-5c69c1f752db admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:19.484514 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3cff7-5d08-47f0-8a85-5c69c1f752db admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:19.484786 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3cff7-5d08-47f0-8a85-5c69c1f752db admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:19.485480 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef3cff7-5d08-47f0-8a85-5c69c1f752db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:19.487039 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef3cff7-5d08-47f0-8a85-5c69c1f752db admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:19.497218 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1432/2859] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:12:19.501118 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7893ff-3118-4d50-9c21-9861583a75a0 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:19.512118 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7893ff-3118-4d50-9c21-9861583a75a0 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:19.517880 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf9e58-b534-4d06-8d0b-1f53858e1073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:19.518500 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf9e58-b534-4d06-8d0b-1f53858e1073 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:19.518774 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf9e58-b534-4d06-8d0b-1f53858e1073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:19.519050 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf9e58-b534-4d06-8d0b-1f53858e1073 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:19.519788 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaf9e58-b534-4d06-8d0b-1f53858e1073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:19.521399 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaf9e58-b534-4d06-8d0b-1f53858e1073 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:19.532499 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1428/2860] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:12:19.537265 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3cff7-5d08-47f0-8a85-5c69c1f752db admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:19.553030 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3cff7-5d08-47f0-8a85-5c69c1f752db admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:19.560211 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0363f35-717a-47bd-83f2-abff06ded2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:19.560908 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0363f35-717a-47bd-83f2-abff06ded2ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:19.561186 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0363f35-717a-47bd-83f2-abff06ded2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:19.561459 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0363f35-717a-47bd-83f2-abff06ded2ba admin admin] PATH_INFO: `/v3/projects/487ac0c07ad444ffa66858feeba7bcee/users/cee98f2141b64b27a00361cd1b232e6c/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:19.577141 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0363f35-717a-47bd-83f2-abff06ded2ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=487ac0c07ad444ffa66858feeba7bcee, user_id=cee98f2141b64b27a00361cd1b232e6c, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:19.579835 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0363f35-717a-47bd-83f2-abff06ded2ba admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:19.595757 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1433/2861] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:12:19 2022] PUT /identity/v3/projects/487ac0c07ad444ffa66858feeba7bcee/users/cee98f2141b64b27a00361cd1b232e6c/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:12:19.607301 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748be2ec-fd3c-4063-bcd4-51e3a0f4d653 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:19.607607 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748be2ec-fd3c-4063-bcd4-51e3a0f4d653 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:19.607891 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748be2ec-fd3c-4063-bcd4-51e3a0f4d653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:19.637120 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-748be2ec-fd3c-4063-bcd4-51e3a0f4d653 None None] MFA Rules not processed for user `cee98f2141b64b27a00361cd1b232e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:12:19.638331 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-748be2ec-fd3c-4063-bcd4-51e3a0f4d653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:19.705671 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1429/2862] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:12:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:12:19.716823 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0363f35-717a-47bd-83f2-abff06ded2ba admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:19.718775 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d0363f35-717a-47bd-83f2-abff06ded2ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:19.777790 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0363f35-717a-47bd-83f2-abff06ded2ba admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:19.781769 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe21bc-6bb7-495c-8f0b-3f165877e065 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:19.782395 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe21bc-6bb7-495c-8f0b-3f165877e065 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:19.782667 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe21bc-6bb7-495c-8f0b-3f165877e065 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:19.782954 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe21bc-6bb7-495c-8f0b-3f165877e065 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:19.792053 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babe21bc-6bb7-495c-8f0b-3f165877e065 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:19.793475 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babe21bc-6bb7-495c-8f0b-3f165877e065 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:19.802940 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1434/2863] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:12:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:12:21.168859 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd7cbc-eb90-4304-b08e-1db024d09d2b None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:21.169406 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd7cbc-eb90-4304-b08e-1db024d09d2b None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:21.169406 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd7cbc-eb90-4304-b08e-1db024d09d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:21.188815 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-21bd7cbc-eb90-4304-b08e-1db024d09d2b None None] MFA Rules not processed for user `cee98f2141b64b27a00361cd1b232e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:12:21.190061 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-21bd7cbc-eb90-4304-b08e-1db024d09d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:21.197438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1430/2864] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:12:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:12:21.670666 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe21bc-6bb7-495c-8f0b-3f165877e065 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:21.672491 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-babe21bc-6bb7-495c-8f0b-3f165877e065 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:21.697956 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:12:21.753726 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe21bc-6bb7-495c-8f0b-3f165877e065 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:21.758650 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef547f33-3a63-4af4-a662-dbad3bc443b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5049dba7ffde47b8af3e6fd55b5703e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:21.759250 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef547f33-3a63-4af4-a662-dbad3bc443b9 service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:21.759534 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef547f33-3a63-4af4-a662-dbad3bc443b9 service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:21.759833 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef547f33-3a63-4af4-a662-dbad3bc443b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:21.761764 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ef547f33-3a63-4af4-a662-dbad3bc443b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:21.773981 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef547f33-3a63-4af4-a662-dbad3bc443b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:21.775544 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef547f33-3a63-4af4-a662-dbad3bc443b9 service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:21.785709 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1435/2865] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:12:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:12:24.352346 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd7cbc-eb90-4304-b08e-1db024d09d2b None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:24.353909 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-21bd7cbc-eb90-4304-b08e-1db024d09d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:24.540504 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd7cbc-eb90-4304-b08e-1db024d09d2b None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:24.560799 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811c643-7a94-40a9-abd6-1a2707859fed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2faeed9002ee4154bc980c146cd06bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:24.562178 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811c643-7a94-40a9-abd6-1a2707859fed service nova] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:24.562633 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811c643-7a94-40a9-abd6-1a2707859fed service nova] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:24.562919 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811c643-7a94-40a9-abd6-1a2707859fed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:24.567398 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-2811c643-7a94-40a9-abd6-1a2707859fed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:24.592522 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2811c643-7a94-40a9-abd6-1a2707859fed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:24.594208 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2811c643-7a94-40a9-abd6-1a2707859fed service nova] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:24.614416 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1431/2866] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:12:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:12:31.781229 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=24 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:12:34.606167 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=90,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:12:36.164953 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef547f33-3a63-4af4-a662-dbad3bc443b9 service placement] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:36.166322 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ef547f33-3a63-4af4-a662-dbad3bc443b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:36.240767 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef547f33-3a63-4af4-a662-dbad3bc443b9 service placement] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:36.245386 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee17e24c-dbdb-4041-aac5-aa0448d976cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:36.246092 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee17e24c-dbdb-4041-aac5-aa0448d976cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:36.246379 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee17e24c-dbdb-4041-aac5-aa0448d976cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:36.246661 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee17e24c-dbdb-4041-aac5-aa0448d976cf admin admin] PATH_INFO: `/v3/users/bca4c30c05b1498f861242024880f2e4` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:36.247917 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee17e24c-dbdb-4041-aac5-aa0448d976cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=bca4c30c05b1498f861242024880f2e4)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:36.249865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee17e24c-dbdb-4041-aac5-aa0448d976cf admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:36.300360 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ee17e24c-dbdb-4041-aac5-aa0448d976cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bca4c30c05b1498f861242024880f2e4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:36.307307 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ee17e24c-dbdb-4041-aac5-aa0448d976cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bca4c30c05b1498f861242024880f2e4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:36.307987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ee17e24c-dbdb-4041-aac5-aa0448d976cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bca4c30c05b1498f861242024880f2e4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:36.318804 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ee17e24c-dbdb-4041-aac5-aa0448d976cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bca4c30c05b1498f861242024880f2e4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:36.319566 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ee17e24c-dbdb-4041-aac5-aa0448d976cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bca4c30c05b1498f861242024880f2e4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:36.337661 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1436/2867] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:12:36 2022] DELETE /identity/v3/users/bca4c30c05b1498f861242024880f2e4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:12:36.497807 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d49e9e-f55c-4474-b9a6-f13d29bd0d7c None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:36.498385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d49e9e-f55c-4474-b9a6-f13d29bd0d7c None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:36.498385 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d49e9e-f55c-4474-b9a6-f13d29bd0d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:36.520651 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee17e24c-dbdb-4041-aac5-aa0448d976cf admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:36.522347 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ee17e24c-dbdb-4041-aac5-aa0448d976cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:36.554812 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-e6d49e9e-f55c-4474-b9a6-f13d29bd0d7c None None] MFA Rules not processed for user `3a8f3eb811ae40ad9cb64261f95a992e`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:12:36.555981 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-e6d49e9e-f55c-4474-b9a6-f13d29bd0d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:36.593328 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee17e24c-dbdb-4041-aac5-aa0448d976cf admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:36.599205 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8b8ac-e390-44b7-a08e-4bf191c65b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:36.599910 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8b8ac-e390-44b7-a08e-4bf191c65b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:36.600190 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8b8ac-e390-44b7-a08e-4bf191c65b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:36.600461 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8b8ac-e390-44b7-a08e-4bf191c65b35 admin admin] PATH_INFO: `/v3/projects/666f12ad571f45c096947d0497291842` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:36.601661 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8b8ac-e390-44b7-a08e-4bf191c65b35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=666f12ad571f45c096947d0497291842)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:36.603627 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8b8ac-e390-44b7-a08e-4bf191c65b35 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:36.621870 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1432/2868] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jul 26 05:12:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:12:36.627797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:12:36.654466 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f2b8b8ac-e390-44b7-a08e-4bf191c65b35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '666f12ad571f45c096947d0497291842', 'request_id': 'req-f2b8b8ac-e390-44b7-a08e-4bf191c65b35'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:36.661045 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f2b8b8ac-e390-44b7-a08e-4bf191c65b35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '666f12ad571f45c096947d0497291842', 'request_id': 'req-f2b8b8ac-e390-44b7-a08e-4bf191c65b35'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:36.669431 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f2b8b8ac-e390-44b7-a08e-4bf191c65b35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '666f12ad571f45c096947d0497291842', 'request_id': 'req-f2b8b8ac-e390-44b7-a08e-4bf191c65b35'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:36.673358 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d49e9e-f55c-4474-b9a6-f13d29bd0d7c None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:36.675152 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-e6d49e9e-f55c-4474-b9a6-f13d29bd0d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:36.680472 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f2b8b8ac-e390-44b7-a08e-4bf191c65b35 admin admin] The token cache is being invalidate because project 666f12ad571f45c096947d0497291842 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:12:36.680762 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f2b8b8ac-e390-44b7-a08e-4bf191c65b35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:36.681398 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f2b8b8ac-e390-44b7-a08e-4bf191c65b35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:36.682933 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1437/2869] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:12:36 2022] DELETE /identity/v3/projects/666f12ad571f45c096947d0497291842 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:12:36.687174 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8b8ac-e390-44b7-a08e-4bf191c65b35 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:36.688527 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f2b8b8ac-e390-44b7-a08e-4bf191c65b35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:36.754874 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8b8ac-e390-44b7-a08e-4bf191c65b35 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:36.759660 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795d25b-df5b-41da-bf67-48a8b27f44f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:36.760317 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795d25b-df5b-41da-bf67-48a8b27f44f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:36.760594 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795d25b-df5b-41da-bf67-48a8b27f44f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:36.760874 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795d25b-df5b-41da-bf67-48a8b27f44f8 admin admin] PATH_INFO: `/v3/users/189407887dd343ecac794fcbf7cbf187` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:36.762045 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795d25b-df5b-41da-bf67-48a8b27f44f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=189407887dd343ecac794fcbf7cbf187)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:36.764014 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795d25b-df5b-41da-bf67-48a8b27f44f8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:36.773211 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d49e9e-f55c-4474-b9a6-f13d29bd0d7c None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:36.811012 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1795d25b-df5b-41da-bf67-48a8b27f44f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '189407887dd343ecac794fcbf7cbf187'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:36.818042 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1795d25b-df5b-41da-bf67-48a8b27f44f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '189407887dd343ecac794fcbf7cbf187'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:36.818797 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1795d25b-df5b-41da-bf67-48a8b27f44f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '189407887dd343ecac794fcbf7cbf187'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:36.824000 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=100 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:12:36.827735 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1795d25b-df5b-41da-bf67-48a8b27f44f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '189407887dd343ecac794fcbf7cbf187'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:36.828430 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1795d25b-df5b-41da-bf67-48a8b27f44f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '189407887dd343ecac794fcbf7cbf187'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:36.835366 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ed93f-abe3-4427-a422-8769e8d13f71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5218319aaa5424882714bffcbeee730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:36.836028 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ed93f-abe3-4427-a422-8769e8d13f71 service swift] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:36.836312 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ed93f-abe3-4427-a422-8769e8d13f71 service swift] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:36.836811 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ed93f-abe3-4427-a422-8769e8d13f71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:36.838466 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-0a6ed93f-abe3-4427-a422-8769e8d13f71 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:36.846535 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1438/2870] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:12:36 2022] DELETE /identity/v3/users/189407887dd343ecac794fcbf7cbf187 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:12:36.895250 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ed93f-abe3-4427-a422-8769e8d13f71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:36.896923 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ed93f-abe3-4427-a422-8769e8d13f71 service swift] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:36.905445 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1433/2871] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jul 26 05:12:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 05:12:37.025887 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795d25b-df5b-41da-bf67-48a8b27f44f8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:37.027474 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1795d25b-df5b-41da-bf67-48a8b27f44f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:37.063987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:12:37.101704 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795d25b-df5b-41da-bf67-48a8b27f44f8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:37.106367 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee75e79-db37-4465-9a54-98813b567bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:37.107025 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee75e79-db37-4465-9a54-98813b567bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:37.107311 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee75e79-db37-4465-9a54-98813b567bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:37.107613 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee75e79-db37-4465-9a54-98813b567bcc admin admin] PATH_INFO: `/v3/projects/a6e3bf6c6e9641e58d936dd8e15550ff` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:37.108795 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee75e79-db37-4465-9a54-98813b567bcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6e3bf6c6e9641e58d936dd8e15550ff)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:37.110742 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee75e79-db37-4465-9a54-98813b567bcc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:37.158819 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6ee75e79-db37-4465-9a54-98813b567bcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6e3bf6c6e9641e58d936dd8e15550ff', 'request_id': 'req-6ee75e79-db37-4465-9a54-98813b567bcc'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:37.165356 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6ee75e79-db37-4465-9a54-98813b567bcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6e3bf6c6e9641e58d936dd8e15550ff', 'request_id': 'req-6ee75e79-db37-4465-9a54-98813b567bcc'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:37.174056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6ee75e79-db37-4465-9a54-98813b567bcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6e3bf6c6e9641e58d936dd8e15550ff', 'request_id': 'req-6ee75e79-db37-4465-9a54-98813b567bcc'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:37.186885 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6ee75e79-db37-4465-9a54-98813b567bcc admin admin] The token cache is being invalidate because project a6e3bf6c6e9641e58d936dd8e15550ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:12:37.187258 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6ee75e79-db37-4465-9a54-98813b567bcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:37.188208 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6ee75e79-db37-4465-9a54-98813b567bcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:37.189740 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1439/2872] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:12:37 2022] DELETE /identity/v3/projects/a6e3bf6c6e9641e58d936dd8e15550ff => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:12:37.449240 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c13ed-49bb-45c5-89c3-9f289e5ce575 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:37.449622 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c13ed-49bb-45c5-89c3-9f289e5ce575 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:37.449965 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c13ed-49bb-45c5-89c3-9f289e5ce575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:37.471287 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-838c13ed-49bb-45c5-89c3-9f289e5ce575 None None] MFA Rules not processed for user `3a8f3eb811ae40ad9cb64261f95a992e`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:12:37.472590 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-838c13ed-49bb-45c5-89c3-9f289e5ce575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:37.537207 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1434/2873] 10.222.0.17 () {58 vars in 967 bytes} [Tue Jul 26 05:12:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:12:37.549718 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee75e79-db37-4465-9a54-98813b567bcc admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:37.551400 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6ee75e79-db37-4465-9a54-98813b567bcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:37.625521 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee75e79-db37-4465-9a54-98813b567bcc admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:37.630987 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b353f7-cb13-4641-85ff-e9ad6ab3ee43 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5218319aaa5424882714bffcbeee730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:37.631602 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b353f7-cb13-4641-85ff-e9ad6ab3ee43 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:37.631905 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b353f7-cb13-4641-85ff-e9ad6ab3ee43 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:37.632180 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b353f7-cb13-4641-85ff-e9ad6ab3ee43 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:37.640355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b353f7-cb13-4641-85ff-e9ad6ab3ee43 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:37.641960 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b353f7-cb13-4641-85ff-e9ad6ab3ee43 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:37.650827 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1440/2874] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jul 26 05:12:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 05:12:37.877271 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c941f1-b045-4e1a-90b7-2b6544147587 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:37.877477 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c941f1-b045-4e1a-90b7-2b6544147587 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:37.877777 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c941f1-b045-4e1a-90b7-2b6544147587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:37.898257 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-27c941f1-b045-4e1a-90b7-2b6544147587 None None] MFA Rules not processed for user `3a8f3eb811ae40ad9cb64261f95a992e`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:12:37.899451 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-27c941f1-b045-4e1a-90b7-2b6544147587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:37.907447 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1435/2875] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jul 26 05:12:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:12:37.923764 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b353f7-cb13-4641-85ff-e9ad6ab3ee43 service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:37.940813 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b353f7-cb13-4641-85ff-e9ad6ab3ee43 service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:37.947716 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4de85d-cece-41bc-92d4-c0888b58d7a6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5218319aaa5424882714bffcbeee730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:37.948341 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4de85d-cece-41bc-92d4-c0888b58d7a6 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:37.948625 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4de85d-cece-41bc-92d4-c0888b58d7a6 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:37.948925 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4de85d-cece-41bc-92d4-c0888b58d7a6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:37.958032 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4de85d-cece-41bc-92d4-c0888b58d7a6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:37.959650 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4de85d-cece-41bc-92d4-c0888b58d7a6 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:37.968333 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1441/2876] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jul 26 05:12:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 05:12:38.306823 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c7a8d-8e34-41c9-81f8-cf930b45b156 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:38.307245 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c7a8d-8e34-41c9-81f8-cf930b45b156 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:38.307755 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c7a8d-8e34-41c9-81f8-cf930b45b156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:38.335996 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-0c9c7a8d-8e34-41c9-81f8-cf930b45b156 None None] MFA Rules not processed for user `3a8f3eb811ae40ad9cb64261f95a992e`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:12:38.337276 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-0c9c7a8d-8e34-41c9-81f8-cf930b45b156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:38.345447 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1436/2877] 10.222.0.17 () {58 vars in 967 bytes} [Tue Jul 26 05:12:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:12:38.358136 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4de85d-cece-41bc-92d4-c0888b58d7a6 service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:38.369256 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4de85d-cece-41bc-92d4-c0888b58d7a6 service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:38.374409 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2070b5-2ace-40cd-97dd-372958e520e2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5218319aaa5424882714bffcbeee730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:38.375019 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2070b5-2ace-40cd-97dd-372958e520e2 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:38.375304 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2070b5-2ace-40cd-97dd-372958e520e2 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:38.375603 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2070b5-2ace-40cd-97dd-372958e520e2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:38.384140 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2070b5-2ace-40cd-97dd-372958e520e2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:38.385671 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2070b5-2ace-40cd-97dd-372958e520e2 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:38.394254 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1442/2878] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jul 26 05:12:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 05:12:40.535619 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c7a8d-8e34-41c9-81f8-cf930b45b156 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:40.537767 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-0c9c7a8d-8e34-41c9-81f8-cf930b45b156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:40.619657 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c7a8d-8e34-41c9-81f8-cf930b45b156 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:40.624688 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21a909-fc1f-46a5-af73-cdbe968e526a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:40.625339 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21a909-fc1f-46a5-af73-cdbe968e526a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:40.625622 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21a909-fc1f-46a5-af73-cdbe968e526a admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:40.625898 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21a909-fc1f-46a5-af73-cdbe968e526a admin admin] PATH_INFO: `/v3/users/cee98f2141b64b27a00361cd1b232e6c` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:40.627150 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21a909-fc1f-46a5-af73-cdbe968e526a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cee98f2141b64b27a00361cd1b232e6c)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:40.629150 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21a909-fc1f-46a5-af73-cdbe968e526a admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:40.664903 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:12:40.678598 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-2b21a909-fc1f-46a5-af73-cdbe968e526a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cee98f2141b64b27a00361cd1b232e6c'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:40.686017 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-2b21a909-fc1f-46a5-af73-cdbe968e526a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cee98f2141b64b27a00361cd1b232e6c'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:40.694993 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-2b21a909-fc1f-46a5-af73-cdbe968e526a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cee98f2141b64b27a00361cd1b232e6c'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:40.695765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-2b21a909-fc1f-46a5-af73-cdbe968e526a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cee98f2141b64b27a00361cd1b232e6c'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:40.696465 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-2b21a909-fc1f-46a5-af73-cdbe968e526a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cee98f2141b64b27a00361cd1b232e6c'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:40.713850 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1437/2879] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:12:40 2022] DELETE /identity/v3/users/cee98f2141b64b27a00361cd1b232e6c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:12:40.718434 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20643c54-7e52-479f-9c37-70edf9a97f9d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:40.718917 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20643c54-7e52-479f-9c37-70edf9a97f9d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:40.719031 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20643c54-7e52-479f-9c37-70edf9a97f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:40.738128 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-20643c54-7e52-479f-9c37-70edf9a97f9d None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:12:40.739304 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-20643c54-7e52-479f-9c37-70edf9a97f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:40.756195 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:12:40.802492 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1443/2880] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:12:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:12:40.813670 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21a909-fc1f-46a5-af73-cdbe968e526a admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:40.816082 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-2b21a909-fc1f-46a5-af73-cdbe968e526a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:40.873958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21a909-fc1f-46a5-af73-cdbe968e526a admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:40.877662 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18358032-9893-4082-9a2f-2e8df4479513 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:40.878281 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18358032-9893-4082-9a2f-2e8df4479513 service neutron] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:40.878560 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18358032-9893-4082-9a2f-2e8df4479513 service neutron] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:40.878836 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18358032-9893-4082-9a2f-2e8df4479513 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:40.887796 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18358032-9893-4082-9a2f-2e8df4479513 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:40.889242 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18358032-9893-4082-9a2f-2e8df4479513 service neutron] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:40.899398 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1438/2881] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:12:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:12:41.094532 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20643c54-7e52-479f-9c37-70edf9a97f9d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:41.096115 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-20643c54-7e52-479f-9c37-70edf9a97f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:41.109752 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20643c54-7e52-479f-9c37-70edf9a97f9d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:41.114858 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d06c4-5a00-4547-95b3-9646c9478928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:41.115618 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d06c4-5a00-4547-95b3-9646c9478928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:41.115995 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d06c4-5a00-4547-95b3-9646c9478928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:41.116369 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d06c4-5a00-4547-95b3-9646c9478928 admin admin] PATH_INFO: `/v3/projects/487ac0c07ad444ffa66858feeba7bcee` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:41.117724 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d06c4-5a00-4547-95b3-9646c9478928 admin admin] RBAC: Authorizing `identity:delete_project(project_id=487ac0c07ad444ffa66858feeba7bcee)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:41.119770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d06c4-5a00-4547-95b3-9646c9478928 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:41.168128 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a44d06c4-5a00-4547-95b3-9646c9478928 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '487ac0c07ad444ffa66858feeba7bcee', 'request_id': 'req-a44d06c4-5a00-4547-95b3-9646c9478928'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:41.175065 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a44d06c4-5a00-4547-95b3-9646c9478928 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '487ac0c07ad444ffa66858feeba7bcee', 'request_id': 'req-a44d06c4-5a00-4547-95b3-9646c9478928'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:41.183946 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a44d06c4-5a00-4547-95b3-9646c9478928 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '487ac0c07ad444ffa66858feeba7bcee', 'request_id': 'req-a44d06c4-5a00-4547-95b3-9646c9478928'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:41.195485 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a44d06c4-5a00-4547-95b3-9646c9478928 admin admin] The token cache is being invalidate because project 487ac0c07ad444ffa66858feeba7bcee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:12:41.195806 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a44d06c4-5a00-4547-95b3-9646c9478928 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:41.196436 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a44d06c4-5a00-4547-95b3-9646c9478928 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:12:41.197969 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1444/2882] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:12:41 2022] DELETE /identity/v3/projects/487ac0c07ad444ffa66858feeba7bcee => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:12:41.203165 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a275f4-4d40-4943-b219-73f998a4f5de None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:41.203515 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a275f4-4d40-4943-b219-73f998a4f5de None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:41.203731 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a275f4-4d40-4943-b219-73f998a4f5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:41.224209 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-42a275f4-4d40-4943-b219-73f998a4f5de None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:12:41.225425 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-42a275f4-4d40-4943-b219-73f998a4f5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:41.285750 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1439/2883] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:12:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 05:12:41.297700 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac2628-0025-40f9-a647-b0dca9ea9aee None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:41.297993 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac2628-0025-40f9-a647-b0dca9ea9aee None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:41.298273 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac2628-0025-40f9-a647-b0dca9ea9aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:41.322407 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-60ac2628-0025-40f9-a647-b0dca9ea9aee None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:12:41.323703 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-60ac2628-0025-40f9-a647-b0dca9ea9aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:41.333166 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1445/2884] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:12:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:12:41.337725 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a275f4-4d40-4943-b219-73f998a4f5de None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:41.348405 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a275f4-4d40-4943-b219-73f998a4f5de None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:41.353639 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ef1c2-e06f-4e14-9d6c-daf92154a461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:41.354286 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ef1c2-e06f-4e14-9d6c-daf92154a461 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:41.354564 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ef1c2-e06f-4e14-9d6c-daf92154a461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:41.354842 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ef1c2-e06f-4e14-9d6c-daf92154a461 admin admin] PATH_INFO: `/v3/domains` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:41.355540 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ef1c2-e06f-4e14-9d6c-daf92154a461 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80205) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 05:12:41.355896 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ef1c2-e06f-4e14-9d6c-daf92154a461 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:41.357499 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ef1c2-e06f-4e14-9d6c-daf92154a461 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:41.369314 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1440/2885] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Jul 26 05:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 05:12:41.373713 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac2628-0025-40f9-a647-b0dca9ea9aee None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:41.383253 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac2628-0025-40f9-a647-b0dca9ea9aee None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:41.389471 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e55d3a-c4f6-41dc-beed-271cfb538156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:41.390106 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e55d3a-c4f6-41dc-beed-271cfb538156 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:41.390387 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e55d3a-c4f6-41dc-beed-271cfb538156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:41.390657 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e55d3a-c4f6-41dc-beed-271cfb538156 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:41.391355 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e55d3a-c4f6-41dc-beed-271cfb538156 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:41.393607 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e55d3a-c4f6-41dc-beed-271cfb538156 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:41.424801 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1446/2886] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Jul 26 05:12:41 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 05:12:41.429182 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ef1c2-e06f-4e14-9d6c-daf92154a461 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:41.440375 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ef1c2-e06f-4e14-9d6c-daf92154a461 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:41.474998 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:12:41.501117 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d0ce6-82bc-462e-be15-be50ced02ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:41.501740 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d0ce6-82bc-462e-be15-be50ced02ed2 admin admin] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:41.502023 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d0ce6-82bc-462e-be15-be50ced02ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:41.502294 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d0ce6-82bc-462e-be15-be50ced02ed2 admin admin] PATH_INFO: `/v3/users` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:41.502955 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d0ce6-82bc-462e-be15-be50ced02ed2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:41.504948 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d0ce6-82bc-462e-be15-be50ced02ed2 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:41.527693 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1441/2887] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jul 26 05:12:41 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 05:12:41.532235 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e55d3a-c4f6-41dc-beed-271cfb538156 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:41.542964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e55d3a-c4f6-41dc-beed-271cfb538156 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:41.547974 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e7778-47fe-4638-8e23-60269d44cc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:41.548598 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e7778-47fe-4638-8e23-60269d44cc7f admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:41.548882 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e7778-47fe-4638-8e23-60269d44cc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:41.549173 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e7778-47fe-4638-8e23-60269d44cc7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:41.549865 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e7778-47fe-4638-8e23-60269d44cc7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:41.551468 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e7778-47fe-4638-8e23-60269d44cc7f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:41.562088 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1447/2888] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:12:41.565994 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d0ce6-82bc-462e-be15-be50ced02ed2 admin admin] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:41.576375 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d0ce6-82bc-462e-be15-be50ced02ed2 admin admin] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:41.581828 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b157c1-bb3b-4c52-b5db-cc4822ba3619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:41.582438 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b157c1-bb3b-4c52-b5db-cc4822ba3619 admin admin] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:41.582727 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b157c1-bb3b-4c52-b5db-cc4822ba3619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:41.583000 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b157c1-bb3b-4c52-b5db-cc4822ba3619 admin admin] PATH_INFO: `/v3/roles` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:41.583823 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b157c1-bb3b-4c52-b5db-cc4822ba3619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:41.585427 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b157c1-bb3b-4c52-b5db-cc4822ba3619 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:41.596549 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1442/2889] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Jul 26 05:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 05:12:41.600994 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e7778-47fe-4638-8e23-60269d44cc7f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:41.611077 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e7778-47fe-4638-8e23-60269d44cc7f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:41.616050 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942321b-2342-4bcf-845e-8292e93e136c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:41.616715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942321b-2342-4bcf-845e-8292e93e136c admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:41.617004 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942321b-2342-4bcf-845e-8292e93e136c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:41.617271 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942321b-2342-4bcf-845e-8292e93e136c admin admin] PATH_INFO: `/v3/projects/1ef8f9da4d8148aebe552f9733dbc45a/users/832161ae82a74a03b6ad578386ed0bb4/roles/7519356859fb40508c25bec8161207fd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:41.632915 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5942321b-2342-4bcf-845e-8292e93e136c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ef8f9da4d8148aebe552f9733dbc45a, user_id=832161ae82a74a03b6ad578386ed0bb4, role_id=7519356859fb40508c25bec8161207fd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:41.635717 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5942321b-2342-4bcf-845e-8292e93e136c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:41.651345 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1448/2890] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Jul 26 05:12:41 2022] PUT /identity/v3/projects/1ef8f9da4d8148aebe552f9733dbc45a/users/832161ae82a74a03b6ad578386ed0bb4/roles/7519356859fb40508c25bec8161207fd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 05:12:41.657418 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de6f2d2-4577-4b0a-b9ef-74c7dc8f7de9 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:41.657803 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de6f2d2-4577-4b0a-b9ef-74c7dc8f7de9 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:41.658139 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de6f2d2-4577-4b0a-b9ef-74c7dc8f7de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:41.680373 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-2de6f2d2-4577-4b0a-b9ef-74c7dc8f7de9 None None] MFA Rules not processed for user `1baa40dae37f4b3298a95bdc939c2f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:12:41.681595 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-2de6f2d2-4577-4b0a-b9ef-74c7dc8f7de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:41.743296 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1443/2891] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:12:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:12:41.753330 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942321b-2342-4bcf-845e-8292e93e136c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:41.755476 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5942321b-2342-4bcf-845e-8292e93e136c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:41.803853 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:12:41.818793 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942321b-2342-4bcf-845e-8292e93e136c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:41.824377 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3419f1b-d175-4704-9cab-2142a536109d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:41.824861 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3419f1b-d175-4704-9cab-2142a536109d service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:41.825147 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3419f1b-d175-4704-9cab-2142a536109d service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:41.825421 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3419f1b-d175-4704-9cab-2142a536109d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:41.835637 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3419f1b-d175-4704-9cab-2142a536109d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:41.837084 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3419f1b-d175-4704-9cab-2142a536109d service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:41.846852 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1449/2892] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:12:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:12:42.509063 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979b22e-b4e7-40e3-8d11-3235d801002a None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:42.509421 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979b22e-b4e7-40e3-8d11-3235d801002a None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:42.509773 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979b22e-b4e7-40e3-8d11-3235d801002a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:42.538989 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-5979b22e-b4e7-40e3-8d11-3235d801002a None None] MFA Rules not processed for user `832161ae82a74a03b6ad578386ed0bb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:12:42.540267 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-5979b22e-b4e7-40e3-8d11-3235d801002a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:42.608351 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1444/2893] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:12:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:12:42.619502 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3419f1b-d175-4704-9cab-2142a536109d service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:12:42.620964 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a3419f1b-d175-4704-9cab-2142a536109d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:42.681374 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3419f1b-d175-4704-9cab-2142a536109d service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:12:42.685848 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2952ec-ac16-4a7a-ae1d-c5389f83ba01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79ef3d83ac454fac9f890374cd22507a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:12:42.686480 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2952ec-ac16-4a7a-ae1d-c5389f83ba01 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:42.686768 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2952ec-ac16-4a7a-ae1d-c5389f83ba01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:42.687040 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2952ec-ac16-4a7a-ae1d-c5389f83ba01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:42.695665 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2952ec-ac16-4a7a-ae1d-c5389f83ba01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:12:42.697140 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2952ec-ac16-4a7a-ae1d-c5389f83ba01 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:12:42.705996 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1450/2894] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:12:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:12:44.070249 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9c392-4473-458f-8773-384e8d5bc684 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:12:44.070798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9c392-4473-458f-8773-384e8d5bc684 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:12:44.070798 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9c392-4473-458f-8773-384e8d5bc684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:12:44.091334 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-8ab9c392-4473-458f-8773-384e8d5bc684 None None] MFA Rules not processed for user `832161ae82a74a03b6ad578386ed0bb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:12:44.092554 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-8ab9c392-4473-458f-8773-384e8d5bc684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:12:44.100393 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1445/2895] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:12:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:12:52.701917 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=33 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:12:54.085966 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=79,keystone:INSERT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:13:33.506076 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2952ec-ac16-4a7a-ae1d-c5389f83ba01 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:13:33.507715 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cc2952ec-ac16-4a7a-ae1d-c5389f83ba01 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:13:33.580264 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2952ec-ac16-4a7a-ae1d-c5389f83ba01 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:13:33.585725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1063809-95af-462e-ba9e-85ea512bb843 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5049dba7ffde47b8af3e6fd55b5703e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:13:33.586518 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1063809-95af-462e-ba9e-85ea512bb843 service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:13:33.586864 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1063809-95af-462e-ba9e-85ea512bb843 service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:13:33.587209 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1063809-95af-462e-ba9e-85ea512bb843 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:13:33.589153 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b1063809-95af-462e-ba9e-85ea512bb843 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:13:33.601725 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1063809-95af-462e-ba9e-85ea512bb843 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:13:33.603323 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1063809-95af-462e-ba9e-85ea512bb843 service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:13:33.614268 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1451/2896] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:13:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:13:42.565262 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d6ebc-b204-4602-b363-9df94ff7ca89 None None] REQUEST_METHOD: `POST` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:13:42.565815 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d6ebc-b204-4602-b363-9df94ff7ca89 None None] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:13:42.565815 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d6ebc-b204-4602-b363-9df94ff7ca89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:13:42.585156 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.auth.core [None req-906d6ebc-b204-4602-b363-9df94ff7ca89 None None] MFA Rules not processed for user `832161ae82a74a03b6ad578386ed0bb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80205) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 05:13:42.586382 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-906d6ebc-b204-4602-b363-9df94ff7ca89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:13:42.593765 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1446/2897] 10.222.0.17 () {58 vars in 958 bytes} [Tue Jul 26 05:13:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 05:13:42.828329 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1063809-95af-462e-ba9e-85ea512bb843 service placement] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:13:42.839520 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1063809-95af-462e-ba9e-85ea512bb843 service placement] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:13:42.844290 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19f6b2-db3f-432b-af18-7d41a0f1b4b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbdb8997891241e9b2e829feb8023f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:13:42.844912 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19f6b2-db3f-432b-af18-7d41a0f1b4b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:13:42.845226 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19f6b2-db3f-432b-af18-7d41a0f1b4b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:13:42.845500 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19f6b2-db3f-432b-af18-7d41a0f1b4b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:13:42.847911 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0f19f6b2-db3f-432b-af18-7d41a0f1b4b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:13:42.858058 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f19f6b2-db3f-432b-af18-7d41a0f1b4b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:13:42.859493 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f19f6b2-db3f-432b-af18-7d41a0f1b4b0 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:13:42.867717 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1452/2898] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:13:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:13:52.580103 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=3 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:13:52.864152 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=38 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:14:38.820957 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d6ebc-b204-4602-b363-9df94ff7ca89 None None] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:14:38.822476 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-906d6ebc-b204-4602-b363-9df94ff7ca89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:14:38.837835 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d6ebc-b204-4602-b363-9df94ff7ca89 None None] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:14:38.843192 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eac9f2-2f6e-49c7-9829-3a75209904c9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5049dba7ffde47b8af3e6fd55b5703e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:14:38.843847 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eac9f2-2f6e-49c7-9829-3a75209904c9 service placement] REQUEST_METHOD: `GET` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:14:38.844133 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eac9f2-2f6e-49c7-9829-3a75209904c9 service placement] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:14:38.844408 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eac9f2-2f6e-49c7-9829-3a75209904c9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:14:38.846251 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.fernet_utils [None req-f6eac9f2-2f6e-49c7-9829-3a75209904c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80205) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:14:38.857958 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eac9f2-2f6e-49c7-9829-3a75209904c9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:14:38.859593 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eac9f2-2f6e-49c7-9829-3a75209904c9 service placement] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:14:38.870716 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1447/2899] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:14:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:14:38.878413 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19f6b2-db3f-432b-af18-7d41a0f1b4b0 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:14:38.891904 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19f6b2-db3f-432b-af18-7d41a0f1b4b0 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:14:38.903271 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab999e2-8377-4626-907f-d63343d5f089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:14:38.904136 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab999e2-8377-4626-907f-d63343d5f089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:14:38.904513 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab999e2-8377-4626-907f-d63343d5f089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:14:38.905026 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab999e2-8377-4626-907f-d63343d5f089 admin admin] PATH_INFO: `/v3/users/832161ae82a74a03b6ad578386ed0bb4` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:14:38.906059 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab999e2-8377-4626-907f-d63343d5f089 admin admin] RBAC: Authorizing `identity:delete_user(user_id=832161ae82a74a03b6ad578386ed0bb4)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:14:38.908045 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab999e2-8377-4626-907f-d63343d5f089 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:14:38.955923 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5ab999e2-8377-4626-907f-d63343d5f089 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '832161ae82a74a03b6ad578386ed0bb4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:14:38.963086 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5ab999e2-8377-4626-907f-d63343d5f089 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '832161ae82a74a03b6ad578386ed0bb4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:14:38.963836 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5ab999e2-8377-4626-907f-d63343d5f089 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '832161ae82a74a03b6ad578386ed0bb4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:14:38.971374 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5ab999e2-8377-4626-907f-d63343d5f089 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '832161ae82a74a03b6ad578386ed0bb4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:14:38.972068 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5ab999e2-8377-4626-907f-d63343d5f089 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '832161ae82a74a03b6ad578386ed0bb4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:14:38.990050 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1453/2900] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Jul 26 05:14:38 2022] DELETE /identity/v3/users/832161ae82a74a03b6ad578386ed0bb4 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:14:39.170713 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eac9f2-2f6e-49c7-9829-3a75209904c9 service placement] Authenticating user token {{(pid=80205) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:14:39.181570 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eac9f2-2f6e-49c7-9829-3a75209904c9 service placement] Validating token access rules against request {{(pid=80205) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:14:39.238261 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7581e-a9e8-4aa9-802a-2759f599b591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1baa40dae37f4b3298a95bdc939c2f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd12823499382429e9825c57ea3292c56', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80205) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:14:39.238914 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7581e-a9e8-4aa9-802a-2759f599b591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:14:39.239193 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7581e-a9e8-4aa9-802a-2759f599b591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:14:39.239471 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7581e-a9e8-4aa9-802a-2759f599b591 admin admin] PATH_INFO: `/v3/projects/1ef8f9da4d8148aebe552f9733dbc45a` {{(pid=80205) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:14:39.240743 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a7581e-a9e8-4aa9-802a-2759f599b591 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ef8f9da4d8148aebe552f9733dbc45a)` {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:14:39.242704 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a7581e-a9e8-4aa9-802a-2759f599b591 admin admin] RBAC: Authorization granted {{(pid=80205) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:14:39.290696 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-a0a7581e-a9e8-4aa9-802a-2759f599b591 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ef8f9da4d8148aebe552f9733dbc45a', 'request_id': 'req-a0a7581e-a9e8-4aa9-802a-2759f599b591'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:14:39.299937 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-a0a7581e-a9e8-4aa9-802a-2759f599b591 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ef8f9da4d8148aebe552f9733dbc45a', 'request_id': 'req-a0a7581e-a9e8-4aa9-802a-2759f599b591'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:14:39.306670 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-a0a7581e-a9e8-4aa9-802a-2759f599b591 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ef8f9da4d8148aebe552f9733dbc45a', 'request_id': 'req-a0a7581e-a9e8-4aa9-802a-2759f599b591'} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:14:39.319050 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-a0a7581e-a9e8-4aa9-802a-2759f599b591 admin admin] The token cache is being invalidate because project 1ef8f9da4d8148aebe552f9733dbc45a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80205) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 05:14:39.319336 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-a0a7581e-a9e8-4aa9-802a-2759f599b591 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:14:39.320076 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG keystone.notifications [None req-a0a7581e-a9e8-4aa9-802a-2759f599b591 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80205) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 05:14:39.321736 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: [pid: 80205|app: 0|req: 1448/2901] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Jul 26 05:14:39 2022] DELETE /identity/v3/projects/1ef8f9da4d8148aebe552f9733dbc45a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 05:14:48.986731 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:14:49.315693 dv-cinder-850574-1-iscsi devstack@keystone.service[80205]: DEBUG dbcounter [-] [80205] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80205) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 05:19:13.136056 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab999e2-8377-4626-907f-d63343d5f089 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 05:19:13.137744 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5ab999e2-8377-4626-907f-d63343d5f089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:19:13.286128 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab999e2-8377-4626-907f-d63343d5f089 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 05:19:13.359060 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41897400-4f43-4a5c-a4b9-d71dc1b49f7c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5049dba7ffde47b8af3e6fd55b5703e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd583e0be8ef74766bffd6c3c07af18f1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 05:19:13.359728 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41897400-4f43-4a5c-a4b9-d71dc1b49f7c service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 05:19:13.360012 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41897400-4f43-4a5c-a4b9-d71dc1b49f7c service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 05:19:13.360284 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41897400-4f43-4a5c-a4b9-d71dc1b49f7c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 05:19:13.363136 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-41897400-4f43-4a5c-a4b9-d71dc1b49f7c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 05:19:13.374896 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41897400-4f43-4a5c-a4b9-d71dc1b49f7c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 05:19:13.376519 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41897400-4f43-4a5c-a4b9-d71dc1b49f7c service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 05:19:13.386770 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1454/2902] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jul 26 05:19:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 05:19:23.382383 dv-cinder-850574-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}